Research on Multiple Classification Based on Improved SVM Algorithm for Balanced Binary Decision Tree

Support vector machines (SVMs) are designed to solve the binary classification problems at the beginning, but in the real world, there are a lot of multiclassification cases. The multiclassification methods based on SVM are mainly divided into the direct methods and the indirect methods, in which the indirect methods, which consist of multiple binary classifiers integrated in accordance with certain rules to form the multiclassification model, are the most commonly used multiclassification methods at present. In this paper, an improved multiclassification algorithm based on the balanced binary decision tree is proposed, which is called the IBDT-SVM algorithm. In this algorithm, it considers not only the influence of “between-classes distance” and “class variance” in traditional measures of between-classes separability but also takes “between-classes variance” into consideration and proposes a new improved “between-classes separability measure.” Based on the new “between-classes separability measure,” it finds out the two classes with the largest between-classes separability measure and uses them as the positive and negative samples to train and learn the classifier. After that, according to the principle of the class-grouping-by-majority, the remaining classes are close to these two classes and merged into the positive samples and the negative samples to train SVM classifier again. For the samples with uneven distribution or sparse distribution, this method can avoid the error caused by the shortest canter distance classification method and overcome the “error accumulation” problem existing in traditional binary decision tree to the greatest extent so as to obtain a better classifier. According to the above algorithm, each layer node of the decision tree is traversed until the output classification result is a single-class label. The experimental results show that the IBDT-SVM algorithm proposed in this paper can achieve better classification accuracy and effectiveness for multiple classification problems.

[1]  Dejan Gjorgjevikj,et al.  A Multi-class SVM Classifier Utilizing Binary Decision Tree , 2009, Informatica.

[2]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[3]  Zhonghui Dong,et al.  An Improved Oversampling Algorithm Based on the Samples’ Selection Strategy for Classifying Imbalanced Data , 2019, Mathematical Problems in Engineering.

[4]  Masayuki Numao,et al.  Efficient Decision Trees for Multi–Class Support Vector Machines Using Entropy and Generalization Error Estimation , 2018, Int. J. Appl. Math. Comput. Sci..

[5]  Rameswar Debnath,et al.  A decision based one-against-one method for multi-class support vector machine , 2004, Pattern Analysis and Applications.

[6]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[7]  Kristin P. Bennett,et al.  Multicategory Classification by Support Vector Machines , 1999, Comput. Optim. Appl..

[8]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..

[9]  Daoliang Li,et al.  Original paper: Classification of foreign fibers in cotton lint using machine vision and multi-class support vector machine , 2010 .

[10]  Zhao Chunjiang,et al.  A new SVM multi-class classification algorithm based on balance decision tree , 2011 .

[11]  Jiang Xiao-lin.  Improved SVM decision-tree and its application in remote sensing classification , 2012 .

[12]  Jordi Vitrià,et al.  Discriminant ECOC: a heuristic method for application dependent design of error correcting output codes , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Chen Xiaoou.  A Strategy of Multi-level Classification Based on SVM , 2005 .

[14]  Jiang Wei-xiang.  Pose-varied Face Recognition Based on Neural Network and Hierarchical Support Vector Machines , 2006 .