Structured Features in Naive Bayes Classification

We propose the structured naive Bayes (SNB) classifier, which augments the ubiquitous naive Bayes classifier with structured features. SNB classifiers facilitate the use of complex features, such as combinatorial objects (e.g., graphs, paths and orders) in a general but systematic way. Underlying the SNB classifier is the recently proposed Probabilistic Sentential Decision Diagram (PSDD), which is a tractable representation of probability distributions over structured spaces. We illustrate the utility and generality of the SNB classifier via case studies. First, we show how we can distinguish players of simple games in terms of play style and skill level based purely on observing the games they play. Second, we show how we can detect anomalous paths taken on graphs based purely on observing the paths themselves.

[1]  Adnan Darwiche,et al.  A differential approach to inference in Bayesian networks , 2000, JACM.

[2]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[3]  Greg Hamerly,et al.  Bayesian approaches to failure prediction for disk drives , 2001, ICML.

[4]  Pedro M. Domingos,et al.  Learning Selective Sum-Product Networks , 2014 .

[5]  Michael I. Jordan,et al.  On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes , 2001, NIPS.

[6]  Daniel Lowd,et al.  Learning Sum-Product Networks with Direct and Indirect Variable Interactions , 2014, ICML.

[7]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[8]  Guy Van den Broeck,et al.  Tractable Learning for Structured Probability Spaces: A Case Study in Learning Preference Distributions , 2015, IJCAI.

[9]  Jonathan Schaeffer,et al.  Opponent Modeling in Poker , 1998, AAAI/IAAI.

[10]  Adnan Darwiche,et al.  Dynamic Minimization of Sentential Decision Diagrams , 2013, AAAI.

[11]  Adnan Darwiche,et al.  A Logical Approach to Factoring Belief Networks , 2002, KR.

[12]  Samy Bengio,et al.  Large-Scale Object Classification Using Label Relation Graphs , 2014, ECCV.

[13]  Adnan Darwiche,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence SDD: A New Canonical Representation of Propositional Knowledge Bases , 2022 .

[14]  Adnan Darwiche,et al.  Basing Decisions on Sentences in Decision Diagrams , 2012, AAAI.

[15]  Takeru Inoue,et al.  Graphillion: software library for very large sets of labeled graphs , 2014, International Journal on Software Tools for Technology Transfer.

[16]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[17]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[18]  Pedro M. Domingos,et al.  Discriminative Learning of Sum-Product Networks , 2012, NIPS.

[19]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[20]  C. L. Mallows.  NON-NULL RANKING MODELS. I , 1957 .

[21]  Pedro M. Domingos,et al.  Sum-product networks: A new deep architecture , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[22]  Guy Van den Broeck,et al.  Tractable Learning for Complex Probability Queries , 2015, NIPS.

[23]  Shin-ichi Minato,et al.  Techniques of BDD/ZDD: Brief History and Recent Activity , 2013, IEICE Trans. Inf. Syst..

[24]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[25]  Guy Van den Broeck,et al.  Probabilistic Sentential Decision Diagrams , 2014, KR.

[26]  Guy Van den Broeck,et al.  Probabilistic sentential decision diagrams: Learning with massive logical constraints , 2014, ICML 2014.

[27]  Daniel Lowd,et al.  Learning Markov Networks With Arithmetic Circuits , 2013, AISTATS.

[28]  Guy Van den Broeck,et al.  On the Role of Canonicity in Knowledge Compilation , 2015, AAAI.

[29]  Pedro M. Domingos,et al.  Learning Arithmetic Circuits , 2008, UAI.

[30]  Adnan Darwiche,et al.  On probabilistic inference by weighted model counting , 2008, Artif. Intell..

[31]  Jaideep Chandrashekar,et al.  When Gossip is Good: Distributed Probabilistic Inference for Detection of Slow Network Intrusions , 2006, AAAI.