Support Vector Machine Decision Trees with Rare Event Detection

Model selection and rare event detection are two major problems frequently encountered in support vector learning. This paper proposes a new support vector learning algorithm, to be referred to as Linear Support Vector Machine Decision Tree (LSVM-DT), that solves the two problems. It consists of a binary tree structure with linear support vector machines in all tree nodes and class labels in all leaves. During training, multiple linear hyperplanes are constructed while traversing down the tree. The LSVM-DT is capable of separating both linearly and nonlinearly separable data. The only parameter that needs to be chosen by the user is the regularization parameter C , thus eliminating the model selection problem. Its built-in rare event detection mechanism allows the LSVM-DT to solve classification problems with underrepresented or unproportional classes. This phenomenon occurs within the tree even if the classes initially have equal prior probabilities. Experiments with different data sets show that the LSVM-DT achieves comparable performance with regular decision tree, polynomial, and Gaussian SVMs. The LSVM-DT can be generalized to Support Vector Machine Decision Tree (SVM-DT) by replacing the linear SVM in each node with a nonlinear SVM.