Supervised classification using probabilistic decision graphs

A new model for supervised classification based on probabilistic decision graphs is introduced. A probabilistic decision graph (PDG) is a graphical model that efficiently captures certain context specific independencies that are not easily represented by other graphical models traditionally used for classification, such as the Naive Bayes (NB) or Classification Trees (CT). This means that the PDG model can capture some distributions using fewer parameters than classical models. Two approaches for constructing a PDG for classification are proposed. The first is to directly construct the model from a dataset of labelled data, while the second is to transform a previously obtained Bayesian classifier into a PDG model that can then be refined. These two approaches are compared with a wide range of classical approaches to the supervised classification problem on a number of both real world databases and artificially generated data.

[1]  Peter C. Cheeseman,et al.  Bayesian Classification (AutoClass): Theory and Results , 1996, Advances in Knowledge Discovery and Data Mining.

[2]  E. Feigenbaum,et al.  Computers and Thought , 1963 .

[3]  Umberto Amato,et al.  Localized empirical discriminant analysis , 2008, Comput. Stat. Data Anal..

[4]  Manfred Jaeger,et al.  Probabilistic Classifiers and the Concepts They Recognize , 2003, ICML.

[5]  Manfred Jaeger,et al.  Learning probabilistic decision graphs , 2006, Int. J. Approx. Reason..

[6]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[7]  Elvira: An Environment for Creating and Using Probabilistic Graphical Models , 2002, Probabilistic Graphical Models.

[8]  Amar Ramdane-Cherif,et al.  Data mining based Bayesian networks for best classification , 2006, Comput. Stat. Data Anal..

[9]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[10]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[11]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[12]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[13]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[14]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[15]  Mehran Sahami,et al.  Learning Limited Dependence Bayesian Classifiers , 1996, KDD.

[16]  José Manuel Gutiérrez,et al.  Expert Systems and Probabiistic Network Models , 1996 .

[17]  Manfred Jaeger,et al.  Probabilistic Decision Graphs - Combining Verification And Ai Techniques For Probabilistic Inference , 2002, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[18]  大西 仁,et al.  Pearl, J. (1988, second printing 1991). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan-Kaufmann. , 1994 .

[19]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[20]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[21]  Insuk Sohn,et al.  Classification of gene functions using support vector machine for time-course gene expression data , 2008, Comput. Stat. Data Anal..

[22]  Steffen L. Lauritzen,et al.  Bayesian updating in causal probabilistic networks by local computations , 1990 .

[23]  Marvin Minsky,et al.  Steps toward Artificial Intelligence , 1995, Proceedings of the IRE.

[24]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[25]  Randal E. Bryant,et al.  Symbolic Boolean manipulation with ordered binary-decision diagrams , 1992, CSUR.

[26]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[27]  Peter J. F. Lucas,et al.  Restricted Bayesian Network Structure Learning , 2002, Probabilistic Graphical Models.

[28]  Oded Maler,et al.  On the Representation of Probabilities over Structured Domains , 1999, CAV.

[29]  Søren Holbech Nielsen,et al.  Proceedings of the Second European Workshop on Probabilistic Graphical Models , 2004 .

[30]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[31]  Serafín Moral,et al.  Building classification trees using the total uncertainty criterion , 2003, Int. J. Intell. Syst..