Learning Naive Bayes Classifiers From Attribute Value Taxonomies and Partially Specified Data

Partially specified data are commonplace in many practical applications of machine learning where different instances are described at different levels of precision relative to an attribute value taxonomy (AVT). This paper describes AVT-NBL – a variant of the Naive Bayes Learning algorithm that effectively exploits user-supplied attribute value taxonomies to construct compact and accurate Naive Bayes classifiers from partially specified data. Our experiments with several data sets and AVTs show that AVT-NBL yields classifiers that are substantially more accurate and more compact than those obtained using the standard Naive Bayes learner.

[1]  Joe Suzuki,et al.  Learning Bayesian Belief Networks Based on the MDL Principle : An Efficient Algorithm Using the Branch and Bound Technique , 1999 .

[2]  Andrew McCallum,et al.  Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data , 2004, J. Mach. Learn. Res..

[3]  Michael J. Pazzani,et al.  Beyond Concise and Colorful: Learning Intelligible Rules , 1997, KDD.

[4]  Amanda Clare,et al.  Knowledge Discovery in Multi-label Phenotype Data , 2001, PKDD.

[5]  Ron Kohavi,et al.  Applications of Data Mining to Electronic Commerce , 2000, Springer US.

[6]  Sally I. McClean,et al.  Aggregation of Imprecise and Uncertain Information in Databases , 2001, IEEE Trans. Knowl. Data Eng..

[7]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[8]  Pat Langley,et al.  An Analysis of Bayesian Classifiers , 1992, AAAI.

[9]  Vasant Honavar,et al.  Ontology-Driven Induction of Decision Trees at Multiple Levels of Abstraction , 2002, SARA.

[10]  Vasant Honavar,et al.  Information extraction and integration from heterogeneous, distributed, autonomous information sources - a federated ontology-driven query-centric approach , 2003, Proceedings Fifth IEEE Workshop on Mobile Computing Systems and Applications.

[11]  Vasant Honavar,et al.  A Framework for Learning from Distributed Data Using Sufficient Statistics and Its Application to Learning Decision Trees , 2004, Int. J. Hybrid Intell. Syst..

[12]  Tom M. Mitchell,et al.  Improving Text Classification by Shrinkage in a Hierarchy of Classes , 1998, ICML.

[13]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[14]  Arbee L. P. Chen,et al.  Evaluating Aggregate Operations Over Imprecise Data , 1996, IEEE Trans. Knowl. Data Eng..

[15]  Marie desJardins,et al.  Using Feature Hierarchies in Bayesian Network Learning , 2000, SARA.

[16]  Timothy W. Finin,et al.  A Target Centric Ontology for Intrusion Detection: Using DAML+OIL to Classify Intrusive Behaviors , 2004 .

[17]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[18]  David Haussler,et al.  Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework , 1988, Artif. Intell..

[19]  John F. Sowa,et al.  Knowledge representation: logical, philosophical, and computational foundations , 2000 .

[20]  Naftali Tishby,et al.  Document clustering using word clusters via the information bottleneck method , 2000, SIGIR '00.

[21]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[22]  Daphne Koller,et al.  Hierarchically Classifying Documents Using Very Few Words , 1997, ICML.

[23]  M. Ashburner,et al.  Gene Ontology: tool for the unification of biology , 2000, Nature Genetics.

[24]  Naftali Tishby,et al.  Distributional Clustering of English Words , 1993, ACL.

[25]  Vasant Honavar,et al.  Learning from Attribute Value Taxonomies and Partially Specified Instances , 2003, International Conference on Machine Learning.

[26]  James A. Hendler,et al.  The Semantic Web" in Scientific American , 2001 .

[27]  Michael J. Pazzani,et al.  Learning Hierarchies from Ambiguous Natural Language Data , 1995, ICML.