Classification systems based on rough sets under the belief function framework

In this paper, we present two classification approaches based on Rough Sets (RS) that are able to learn decision rules from uncertain data. We assume that the uncertainty exists only in the decision attribute values of the Decision Table (DT) and is represented by the belief functions. The first technique, named Belief Rough Set Classifier (BRSC), is based only on the basic concepts of the Rough Sets (RS). The second, called Belief Rough Set Classifier, is more sophisticated. It is based on Generalization Distribution Table (BRSC-GDT), which is a hybridization of the Generalization Distribution Table and the Rough Sets (GDT-RS). The two classifiers aim at simplifying the Uncertain Decision Table (UDT) in order to generate significant decision rules for classification process. Furthermore, to improve the time complexity of the construction procedure of the two classifiers, we apply a heuristic method of attribute selection based on rough sets. To evaluate the performance of each classification approach, we carry experiments on a number of standard real-world databases by artificially introducing uncertainty in the decision attribute values. In addition, we test our classifiers on a naturally uncertain web usage database. We compare our belief rough set classifiers with traditional classification methods only for the certain case. Besides, we compare the results relative to the uncertain case with those given by another similar classifier, called the Belief Decision Tree (BDT), which also deals with uncertain decision attribute values.

[1]  Ning Zhong,et al.  Probabilistic Rough Induction: The GDT-RS Methodology and Algorithms , 1999, ISMIS.

[2]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[3]  Ronald P. S. Mahler,et al.  The modified Dempster-Shafer approach to classification , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[4]  Catherine K. Murphy Combining belief functions when evidence conflicts , 2000, Decis. Support Syst..

[5]  Zied Elouedi,et al.  Belief Rough Set Classifier , 2009, Canadian Conference on AI.

[6]  Philippe Smets,et al.  The Transferable Belief Model , 1994, Artif. Intell..

[7]  Z. Pawlak Rough Sets: Theoretical Aspects of Reasoning about Data , 1991 .

[8]  Shusaku Tsumoto,et al.  Foundations of Intelligent Systems, 15th International Symposium, ISMIS 2005, Saratoga Springs, NY, USA, May 25-28, 2005, Proceedings , 2005, ISMIS.

[9]  Tuan-Fang Fan,et al.  Dominance-based fuzzy rough set analysis of uncertain and possibilistic data tables , 2011, Int. J. Approx. Reason..

[10]  Patrick Vannoorenberghe,et al.  On aggregating belief decision trees , 2004, Inf. Fusion.

[11]  P. Smets,et al.  Assessing sensor reliability for multisensor data fusion within the transferable belief model , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  P. Smets Application of the transferable belief model to diagnostic problems , 1998 .

[13]  Roman Słowiński,et al.  Intelligent Decision Support , 1992, Theory and Decision Library.

[14]  Pawan Lingras,et al.  Interval Set Clustering of Web Users with Rough K-Means , 2004, Journal of Intelligent Information Systems.

[15]  Thierry Denoeux,et al.  Induction of decision trees from partially classified data using belief functions , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[16]  Belur V. Dasarathy,et al.  Nearest neighbor (NN) norms: NN pattern classification techniques , 1991 .

[17]  Ning Zhong,et al.  Using Rough Sets with Heuristics for Feature Selection , 1999, Journal of Intelligent Information Systems.

[18]  Jerzy W. Grzymala-Busse,et al.  A Rough Set Approach to Data with Missing Attribute Values , 2006, RSKT.

[19]  Zied Elouedi,et al.  Rule Discovery Process Based on Rough Sets under the Belief Function Framework , 2010, IPMU.

[20]  Éloi Bossé,et al.  A new distance between two bodies of evidence , 2001, Inf. Fusion.

[21]  J. Grzymala-Busse,et al.  Rough Set Approaches to Rule Induction from Incomplete Data , 2004 .

[22]  Pradipta Maji,et al.  Rough set based maximum relevance-maximum significance criterion and Gene selection from microarray data , 2011, Int. J. Approx. Reason..

[23]  Tzung-Pei Hong,et al.  Learning fuzzy rules from incomplete quantitative data by rough sets , 2002, 2002 IEEE World Congress on Computational Intelligence. 2002 IEEE International Conference on Fuzzy Systems. FUZZ-IEEE'02. Proceedings (Cat. No.02CH37291).

[24]  Khaled Mellouli,et al.  Pruning belief decision tree methods in averaging and conjunctive approaches , 2007, Int. J. Approx. Reason..

[25]  Khaled Mellouli,et al.  Clustering Approach Using Belief Function Theory , 2006, AIMSA.

[26]  Xiao-Jun Zeng,et al.  Core-generating approximate minimum entropy discretization for rough set feature selection in pattern classification , 2011, Int. J. Approx. Reason..

[27]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[28]  Jerzy W. Grzymala-Busse,et al.  Rough Set Strategies to Data with Missing Attribute Values , 2006, Foundations and Novel Approaches in Data Mining.

[29]  Zied Elouedi,et al.  Belief Rough Set Classification for web mining based on dynamic core , 2010, 2010 10th International Conference on Intelligent Systems Design and Applications.

[30]  Mathias Bauer,et al.  Approximation algorithms and decision making in the Dempster-Shafer theory of evidence - An empirical study , 1997, Int. J. Approx. Reason..

[31]  Nick Cercone,et al.  Applying Knowledge Discovery to Predict Water-Supply Consumption , 1997, IEEE Expert.

[32]  Philippe Smets,et al.  The Transferable Belief Model for Quantified Belief Representation , 1998 .

[33]  Dov M. Gabbay,et al.  Handbook of defeasible reasoning and uncertainty management systems: volume 2: reasoning with actual and potential contradictions , 1998 .

[34]  Andrzej Skowron,et al.  The Discernibility Matrices and Functions in Information Systems , 1992, Intelligent Decision Support.

[35]  Thierry Denoeux,et al.  An evidence-theoretic k-NN rule with parameter optimization , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[36]  Khaled Mellouli,et al.  Belief decision trees: theoretical foundations , 2001, Int. J. Approx. Reason..

[37]  Zied Elouedi,et al.  Heuristic method for attribute selection from partially uncertain data using rough sets , 2010, Int. J. Gen. Syst..

[38]  Andrzej Skowron,et al.  From the Rough Set Theory to the Evidence Theory , 1991 .

[39]  Thierry Denoeux,et al.  A k-nearest neighbor classification rule based on Dempster-Shafer theory , 1995, IEEE Trans. Syst. Man Cybern..

[40]  Jerzy W. Grzymala-Busse,et al.  Rough Sets , 1995, Commun. ACM.

[41]  Zied Elouedi,et al.  LEARNING DECISION RULES FROM UNCERTAIN DATA USING ROUGH SETS , 2008 .

[42]  Didier Dubois,et al.  Putting Rough Sets and Fuzzy Sets Together , 1992, Intelligent Decision Support.

[43]  N. Zhong,et al.  Data Mining: A Probabilistic Rough Set Approach , 1998 .

[44]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[45]  Thierry Denoeux,et al.  A neural network classifier based on Dempster-Shafer theory , 2000, IEEE Trans. Syst. Man Cybern. Part A.