Probabilistic decision trees using SVM for multi-class classification

In the automotive repairing backdrop, retrieving from previously solved incident database features that could support and speed up the diagnostic is of great usefulness. This decision helping process should give a fixed number of the most relevant diagnostic procedures classified in a likelihood sense. It is a probabilistic multi-class classification problem. This paper describes an original classification technique, PDT (Probabilistic Decision Tree) producing a posteriori probability in a multi-class context. It is based on a binary decision tree (BDT) with probabilistic support vector machine classifier (PSVM). At each node of the tree, a bi-class SVM along with a sigmoid function are trained to give a probabilistic classification output. For each branch, the outputs of all the nodes composing the branch are combined to lead to a complete evaluation of the probability when reaching the final leaf (representing the class associated to the branch). To show the effectiveness of PDTs, they are tested on benchmark datasets and results are compared to other existing approaches.

[1]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[2]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[3]  Chih-Jen Lin,et al.  Asymptotic Behaviors of Support Vector Machines with Gaussian Kernel , 2003, Neural Computation.

[4]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[5]  Zhiyong Hu,et al.  Modeling urban growth in Atlanta using logistic regression , 2007, Comput. Environ. Urban Syst..

[6]  Dejan Gjorgjevikj,et al.  A Multi-class SVM Classifier Utilizing Binary Decision Tree , 2009, Informatica.

[7]  D. Hosmer,et al.  Applied Logistic Regression , 1991 .

[8]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[9]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[10]  Tristan Fletcher,et al.  Support Vector Machines Explained , 2008 .

[11]  Grace Wahba,et al.  Soft and hard classification by reproducing kernel Hilbert space methods , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Hsuan-Tien Lin,et al.  A note on Platt’s probabilistic outputs for support vector machines , 2007, Machine Learning.

[13]  PingSun Leung,et al.  Predicting shrimp disease occurrence: artificial neural networks vs. logistic regression , 2000 .

[14]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[15]  G. Wahba,et al.  Multicategory Support Vector Machines , Theory , and Application to the Classification of Microarray Data and Satellite Radiance Data , 2004 .

[16]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[17]  Hao Helen Zhang,et al.  Hard or Soft Classification? Large-Margin Unified Machines , 2011, Journal of the American Statistical Association.

[18]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[19]  C. V. Jawahar,et al.  Support Vector Machine Based Hierarchical Classifiers for Large Class Problems , 2006 .