Combining Decision Trees Based on Imprecise Probabilities and Uncertainty Measures

In this article, we shall present a method for combining classification trees obtained by a simple method from the imprecise Dirichlet model (IDM) and uncertainty measures on closed and convex sets of probability distributions, otherwise known as credal sets. Our combine method has principally two characteristics: it obtains a high percentage of correct classifications using a few number of classification trees and it can be parallelized to apply on very large databases.

[1]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[2]  Serafín Moral,et al.  Difference of entropies as a non-specificity function on credal sets† , 2005, Int. J. Gen. Syst..

[3]  Yoshua Bengio,et al.  Inference for the Generalization Error , 1999, Machine Learning.

[4]  D. Dubois,et al.  Properties of measures of information in evidence and possibility theories , 1987 .

[5]  Arthur P. Dempster,et al.  Upper and Lower Probabilities Induced by a Multivalued Mapping , 1967, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[6]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[7]  Serafín Moral,et al.  An Algorithm to Compute the Upper Entropy for Order-2 Capacities , 2006, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[8]  S. Moral,et al.  COMPLETING A TOTAL UNCERTAINTY MEASURE IN THE DEMPSTER-SHAFER THEORY , 1999 .

[9]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[10]  P. Walley.  Inferences from Multinomial Data: Learning About a Bag of Marbles , 1996 .

[11]  G. Klir.  Uncertainty and Information: Foundations of Generalized Information Theory , 2005 .

[12]  A. Ramer.  Uniqueness of information measure in the theory of evidence , 1987 .

[13]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[14]  Serafín Moral,et al.  Building classification trees using the total uncertainty criterion , 2003, Int. J. Intell. Syst..

[15]  Joaquín Abellán,et al.  Uncertainty measures on probability intervals from the imprecise Dirichlet model , 2006, Int. J. Gen. Syst..

[16]  Khaled Mellouli,et al.  Belief decision trees: theoretical foundations , 2001, Int. J. Approx. Reason..

[17]  George J. Klir,et al.  On Measuring Uncertainty and Uncertainty-Based Information: Recent Developments , 2001, Annals of Mathematics and Artificial Intelligence.

[18]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[19]  Patrick Vannoorenberghe,et al.  On aggregating belief decision trees , 2004, Inf. Fusion.

[20]  George J. Klir,et al.  Disaggregated total uncertainty measure for credal sets , 2006, Int. J. Gen. Syst..

[21]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[22]  Serafín Moral,et al.  Upper entropy of credal sets. Applications to credal classification , 2005, Int. J. Approx. Reason..

[23]  Jean-Marc Bernard,et al.  An introduction to the imprecise Dirichlet model for multinomial data , 2005, Int. J. Approx. Reason..

[24]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Ian Witten,et al.  Data Mining , 2000 .

[26]  Serafín Moral,et al.  A non-specificity measure for convex sets of probability distributions , 2000 .

[27]  George J. Klir,et al.  Uncertainty-Based Information , 1999 .

[28]  Serafín Moral,et al.  Maximum of Entropy for Credal Sets , 2003, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[29]  P. Walley.  Statistical Reasoning with Imprecise Probabilities , 1990 .