Classification confidence weighted majority voting using decision tree classifiers

Purpose – The purpose of this paper is to provide classification confidence value to every individual sample classified by decision trees and use this value to combine the classifiers.Design/methodology/approach – The proposed system is first theoretically explained, and then the use and effectiveness of the proposed system is demonstrated on sample datasets.Findings – In this paper, a novel method is proposed to combine decision tree classifiers using calculated classification confidence values. This confidence in the classification is based on distance calculation to the relevant decision boundary (distance conditional), probability density estimation and (distance conditional) classification confidence estimation. It is shown that these values – provided by individual classification trees – can be integrated to derive a consensus decision.Research limitations/implications – The proposed method is not limited to axis‐parallel trees, it is applicable not only to oblique trees, but also to any kind of cla...

[1]  Stephan Bernard,et al.  Ranking Cases with Decision Trees: a Geometric Method that Preserves Intelligibility , 2005, IJCAI.

[2]  David G. Stork,et al.  Pattern Classification , 1973 .

[3]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[4]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[5]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Christopher M. Bishop,et al.  Classification and regression , 1997 .

[7]  Alexander G. Gray,et al.  Retrofitting Decision Tree Classifiers Using Kernel Density Estimation , 1995, ICML.

[8]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[9]  Shinichi Morishita,et al.  On Classification and Regression , 1998, Discovery Science.

[10]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[11]  Pedro M. Domingos,et al.  Tree Induction for Probability-Based Ranking , 2003, Machine Learning.

[12]  Michael I. Jordan,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1994, Neural Computation.

[13]  Peter A. Flach,et al.  Decision Trees for Ranking: Effect of new smoothing methods, new splitting criteria and simple pruning methods , 2003 .

[14]  C. Ling,et al.  Decision Tree with Better Ranking , 2003, ICML.

[15]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[16]  Guillaume Deffuant,et al.  Keep the Decision Tree and Estimate the Class Probabilities Using its Decision Boundary , 2007, IJCAI.