Inference in possibilistic network classifiers under uncertain observations

Possibilistic networks, which are compact representations of possibility distributions, are powerful tools for representing and reasoning with uncertain and incomplete information in the framework of possibility theory. They are like Bayesian networks but lie on possibility theory to deal with uncertainty, imprecision and incompleteness. While classification is a very useful task in many real world applications, possibilistic network-based classification issues are not well investigated in general and possibilistic-based classification inference with uncertain observations in particular. In this paper, we address on one hand the theoretical foundations of inference in possibilistic classifiers under uncertain inputs and propose on the other hand a novel efficient algorithm for the inference in possibilistic network-based classification under uncertain observations. We start by studying and analyzing the counterpart of Jeffrey’s rule in the framework of possibility theory. After that, we address the validity of Markov-blanket criterion in the context of possibilistic networks used for classification with uncertain inputs purposes. Finally, we propose a novel algorithm suitable for possibilistic classifiers with uncertain observations without assuming any independence relations between observations. This algorithm guarantees the same results as if classification were performed using the possibilistic counterpart of Jeffrey’s rule. Classification is achieved in polynomial time if the target variable is binary. The basic idea of our algorithm is to only search for totally plausible class instances through a series of equivalent and polynomial transformations applied on the possibilistic classifier taking into account the uncertain observations.

[1]  C. K. Chow,et al.  On optimum recognition error and reject tradeoff , 1970, IEEE Trans. Inf. Theory.

[2]  C. Borgelt,et al.  A Naive Bayes Style Possibilistic Classifier , 2004 .

[3]  Michael I. Jordan,et al.  Probabilistic Networks and Expert Systems , 1999 .

[4]  Gisele L. Pappa,et al.  Exploiting contexts to deal with uncertainty in classification , 2009, U '09.

[5]  Khaled Mellouli,et al.  A Two-Steps Algorithm for Min-Based Possibilistic Causal Networks , 2001, ECSQARU.

[6]  Salem Benferhat,et al.  Alert Correlation based on a Logical Handling of Administrator Preferences and Knowledge , 2018, SECRYPT.

[7]  C. Aitken,et al.  The logic of decision , 2014 .

[8]  Didier Dubois,et al.  Possibility theory , 2018, Scholarpedia.

[9]  Didier Dubois,et al.  A definition of subjective possibility , 2008, Int. J. Approx. Reason..

[10]  Khaled Mellouli,et al.  Naïve possibilistic network classifiers , 2009, Fuzzy Sets Syst..

[11]  Hervé Debar,et al.  Aggregation and Correlation of Intrusion-Detection Alerts , 2001, Recent Advances in Intrusion Detection.

[12]  Sally I. McClean,et al.  Database aggregation of imprecise and uncertain evidence , 2003, Inf. Sci..

[13]  Bernadette Bouchon-Meunier,et al.  Independence and Possibilistic Conditioning , 2000, Annals of Mathematics and Artificial Intelligence.

[14]  Slawomir Zadrozny,et al.  Fuzzy information retrieval model revisited , 2009, Fuzzy Sets Syst..

[15]  Jinbo Bi,et al.  Support Vector Classification with Input Data Uncertainty , 2004, NIPS.

[16]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[17]  Shubha U. Nabar,et al.  Representing Uncertain Data: Uniqueness, Equivalence, Minimization, and Approximation , 2005 .

[18]  Marco Zaffalon,et al.  Lazy naive credal classifier , 2009, U '09.

[19]  L. Zadeh Fuzzy sets as a basis for a theory of possibility , 1999 .

[20]  Ankush Mittal,et al.  Bayesian Network Technologies: Applications and Graphical Models , 2007 .

[21]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[22]  Didier Dubois,et al.  Possibility Theory - An Approach to Computerized Processing of Uncertainty , 1988 .

[23]  Stefan Axelsson,et al.  Intrusion Detection Systems: A Survey and Taxonomy , 2002 .

[24]  Adnan Darwiche,et al.  On the Revision of Probabilistic Beliefs using Uncertain Evidence , 2003, IJCAI.

[25]  Frédéric Cuppens,et al.  Alert correlation in a cooperative intrusion detection framework , 2002, Proceedings 2002 IEEE Symposium on Security and Privacy.

[26]  Philip S. Yu,et al.  A Survey of Uncertain Data Algorithms and Applications , 2009, IEEE Transactions on Knowledge and Data Engineering.

[27]  Peng Ning,et al.  Constructing attack scenarios through correlation of intrusion alerts , 2002, CCS '02.

[28]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..

[29]  Jirí Vomlel,et al.  Soft evidential update for probabilistic multiagent systems , 2002, Int. J. Approx. Reason..

[30]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[31]  Didier Dubois,et al.  A synthetic view of belief revision with uncertain inputs in the framework of possibility theory , 1997, Int. J. Approx. Reason..

[32]  Matthias C. M. Troffaes Decision making under uncertainty using imprecise probabilities , 2007, Int. J. Approx. Reason..

[33]  E. Hisdal Conditional possibilities independence and noninteraction , 1978 .

[34]  Guy De Tré,et al.  The application of fuzzy logic and soft computing in information management , 2009, Fuzzy Sets Syst..

[35]  Jung-Min Park,et al.  An overview of anomaly detection techniques: Existing solutions and latest technological trends , 2007, Comput. Networks.

[36]  Eyke Hüllermeier Possibilistic Induction in Decision-Tree Learning , 2002, ECML.

[37]  Didier Dubois,et al.  Possibility theory and statistical reasoning , 2006, Comput. Stat. Data Anal..

[38]  Khaled Mellouli,et al.  Decision Trees and Qualitative Possibilistic Inference: Application to the Intrusion Detection Problem , 2003, ECSQARU.

[39]  Pavel Brazdil,et al.  Proceedings of the European Conference on Machine Learning , 1993 .

[40]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[41]  Bianca Zadrozny,et al.  Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers , 2001, ICML.

[42]  Thierry Denoeux,et al.  Handling possibilistic labels in pattern classification using evidential reasoning , 2001, Fuzzy Sets Syst..

[43]  Eric V. Slud Graphical Models: Methods for Data Analysis and Mining , 2003, Technometrics.

[44]  Thierry Denoeux,et al.  Inferring a possibility distribution from empirical data , 2006, Fuzzy Sets Syst..

[45]  Pascale Fonck,et al.  A comparative study of possibilistic conditional independence and lack of interaction , 1997, Int. J. Approx. Reason..

[46]  Wolfgang Spohn,et al.  Ordinal Conditional Functions: A Dynamic Theory of Epistemic States , 1988 .

[47]  Christian Borgelt,et al.  Learning from imprecise data: Possibilistic graphical models , 2002 .

[48]  C. J. van Rijsbergen,et al.  Getting into Information Retrieval , 2001, ESSIR.

[49]  Salem Benferhat,et al.  An Efficient Algorithm for Naive Possibilistic Classifiers with Uncertain Inputs , 2008, SUM.

[50]  Simon Maskell,et al.  A Bayesian approach to fusing uncertain, imprecise and conflicting information , 2008, Inf. Fusion.

[51]  Lakhmi C. Jain,et al.  Introduction to Bayesian Networks , 2008 .

[52]  András A. Benczúr,et al.  Web spam detection via commercial intent analysis , 2007, AIRWeb '07.

[53]  Maria Papadaki,et al.  Investigating the problem of IDS false alarms: An experimental study using Snort , 2008, SEC.

[54]  Jiri Vomlel,et al.  Probabilistic reasoning with uncertain evidence , 2004 .

[55]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[56]  Charu C. Aggarwal,et al.  Managing and Mining Uncertain Data , 2009, Advances in Database Systems.