Multi-dimensional Bayesian Network Classifiers

We introduce the family of multi-dimensional Bayesian network classifiers. These classifiers include one or more class variables and multiple feature variables, which need not be modelled as being dependent on every class variable. Our family of multi-dimensional classifiers includes as special cases the well-known naive Bayesian and tree-augmented classifiers, yet offers better modelling capabilities than families of models with a single class variable. We describe the learning problem for a subfamily of multi-dimensional classifiers and show that the complexity of the solution algorithm is polynomial in the number of variables involved. We further present some preliminary experimental results to illustrate the benefits of the multi-dimensionality of our classifiers.

[1]  J. Kruskal.  On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .

[2]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[3]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  Pat Langley,et al.  An Analysis of Bayesian Classifiers , 1992, AAAI.

[6]  Mehran Sahami,et al.  Learning Limited Dependence Bayesian Classifiers , 1996, KDD.

[7]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[8]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[9]  Eamonn J. Keogh,et al.  Learning augmented Bayesian classifiers: A comparison of distribution-based and classification-based approaches , 1999, AISTATS.

[10]  Peter J. F. Lucas,et al.  Restricted Bayesian Network Structure Learning , 2002, Probabilistic Graphical Models.

[11]  James D. Park,et al.  MAP Complexity Results and Approximation Methods , 2002, UAI.

[12]  Linda C. van der Gaag,et al.  On the Complexity of the MPA Problem in Probabilistic Networks , 2002, ECAI.

[13]  Silja Renooij,et al.  Probabilities for a probabilistic network: a case study in oesophageal cancer , 2002, Artif. Intell. Medicine.

[14]  Constantin F. Aliferis,et al.  Towards Principled Feature Selection: Relevancy, Filters and Wrappers , 2003 .

[15]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.