Complement information entropy for uncertainty measure in fuzzy rough set and its applications

Uncertainty measure is an important tool for analyzing imprecise and ambiguous data. Some information entropy models in rough set theory have been defined for various information systems. However, there are relatively few studies on evaluating uncertainty in fuzzy rough set. In this paper, we propose a new complement information entropy model in fuzzy rough set based on arbitrary fuzzy relation, which takes inner-class and outer-class information into consideration. The corresponding definitions of complement conditional entropy, complement joint entropy, complement mutual information and complement information granularity are also presented. The properties of these definitions are analyzed, which show complement information entropy shares some similar properties with Shannon’s entropy. Moreover, a generalized information entropy model is proposed by introducing probability distribution into fuzzy approximate space. This model can be used to measure uncertainty of data with the different sample distributions. Applications of the proposed entropy measures in feature importance evaluation and feature selection are studied with data set experiments. Experimental results show that the proposed method is effective and adaptable to different classifiers.

[1]  Naif Alajlan,et al.  Measure based representation of uncertain information , 2012, Fuzzy Optim. Decis. Mak..

[2]  Ronald R. Yager.  Entailment Principle for Measure-Based Uncertainty , 2012, IEEE Transactions on Fuzzy Systems.

[3]  Yee Leung,et al.  On characterizations of (I, J)-fuzzy rough approximation operators , 2005, Fuzzy Sets Syst..

[4]  L. Zadeh.  Fuzzy sets as a basis for a theory of possibility , 1999 .

[5]  Radko Mesiar,et al.  Fuzzy measures and integrals , 2005, Fuzzy Sets Syst..

[6]  Jiye Liang,et al.  A new measure of uncertainty based on knowledge granulation for rough sets , 2009, Inf. Sci..

[7]  G. Cooman.  POSSIBILITY THEORY I: THE MEASURE- AND INTEGRAL-THEORETIC GROUNDWORK , 1997 .

[8]  Tao Feng,et al.  Information-Theoretic Measure of Uncertainty in Generalized Fuzzy Rough Sets , 2007, RSFDGrC.

[9]  George J. Klir,et al.  On Measuring Uncertainty and Uncertainty-Based Information: Recent Developments , 2001, Annals of Mathematics and Artificial Intelligence.

[10]  Jianhua Dai,et al.  Uncertainty measurement for interval-valued information systems , 2013, Inf. Sci..

[11]  Settimo Termini,et al.  A Definition of a Nonprobabilistic Entropy in the Setting of Fuzzy Sets Theory , 1972, Inf. Control..

[12]  Weihua Xu,et al.  Multi-granulation rough sets based on tolerance relations , 2013, Soft Computing.

[13]  Yuhan Liu,et al.  Uncertain random variables: a mixture of uncertainty and randomness , 2013, Soft Comput..

[14]  M. J. Wierman,et al.  MEASURING UNCERTAINTY IN ROUGH SET THEORY , 1999 .

[15]  Qiang Shen,et al.  Fuzzy-Rough Sets Assisted Attribute Selection , 2007, IEEE Transactions on Fuzzy Systems.

[16]  Romesh Nagarajah,et al.  Uncertainty Estimation Using Fuzzy Measures for Multiclass Classification , 2007, IEEE Transactions on Neural Networks.

[17]  Jianhua Dai,et al.  An Uncertainty Measure for Incomplete Decision Tables and Its Applications , 2013, IEEE Transactions on Cybernetics.

[18]  Qinghua Hu,et al.  Uncertainty measures for fuzzy relations and their applications , 2007, Appl. Soft Comput..

[19]  Zhi-Li Zhang,et al.  Fuzzy-Rough Data Reduction Based on Information Entropy , 2007, 2007 International Conference on Machine Learning and Cybernetics.

[20]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[21]  Wen-Xiu Zhang,et al.  An axiomatic characterization of a fuzzy generalization of rough sets , 2004, Inf. Sci..

[22]  Jiye Liang,et al.  A new method for measuring uncertainty and fuzziness in rough set theory , 2002, Int. J. Gen. Syst..

[23]  Giulianella Coletti,et al.  Possibility Measures in Probabilistic Inference , 2008, SMPS.

[24]  Jiye Liang,et al.  Information entropy, rough entropy and knowledge granulation in incomplete information systems , 2006, Int. J. Gen. Syst..

[25]  D. Dubois,et al.  ROUGH FUZZY SETS AND FUZZY ROUGH SETS , 1990 .

[26]  G.J. Klir.  Measuring uncertainty associated with convex sets of probability distributions: a new approach , 2005, NAFIPS 2005 - 2005 Annual Meeting of the North American Fuzzy Information Processing Society.

[27]  Gert de Cooman,et al.  POSSIBILITY THEORY III: POSSIBILISTIC INDEPENDENCE , 1997 .

[28]  James C. Bezdek,et al.  Measuring fuzzy uncertainty , 1994, IEEE Trans. Fuzzy Syst..

[29]  Enrico Zio,et al.  An imprecision importance measure for uncertainty representations interpreted as lower and upper probabilities, with special emphasis on possibility theory , 2012 .

[30]  Didier Dubois,et al.  Possibility theory , 2018, Scholarpedia.

[31]  Esfandiar Eslami,et al.  Uncertain probabilities I: the discrete case , 2003, Soft Comput..

[32]  Jianwen Chen,et al.  Dealing with Uncertainty: A Survey of Theories and Practices , 2013, IEEE Transactions on Knowledge and Data Engineering.

[33]  Ronald R. Yager,et al.  Uncertainty representation using fuzzy measures , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[34]  Yee Leung,et al.  An uncertainty measure in partition-based fuzzy rough sets , 2005, Int. J. Gen. Syst..

[35]  Esfandiar Eslami,et al.  Uncertain probabilities II: the continuous case , 2004, Soft Comput..

[36]  Qiang Shen,et al.  Linguistic probabilities: theory and application , 2008, Soft Comput..

[37]  L. Zadeh.  Probability measures of Fuzzy events , 1968 .

[38]  Witold Pedrycz,et al.  Gaussian kernel based fuzzy rough sets: Model, uncertainty measures and applications , 2010, Int. J. Approx. Reason..

[39]  Hung T. Nguyen,et al.  Random sets and large deviations principle as a foundation for possibility measures , 2003, Soft Comput..

[40]  Jiye Liang,et al.  A New Method for Measuring the Uncertainty in Incomplete Information Systems , 2009, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[41]  Carlo Bertoluzza,et al.  Uncertainty measure on fuzzy partitions , 2004, Fuzzy Sets Syst..

[42]  Jianhua Dai,et al.  Entropy measures and granularity measures for set-valued information systems , 2013, Inf. Sci..

[43]  Theresa Beaubouef,et al.  Information-Theoretic Measures of Uncertainty for Rough Sets and Rough Relational Databases , 1998, Inf. Sci..

[44]  Ronald R. Yager,et al.  A measure based approach to the fusion of possibilistic and probabilistic uncertainty , 2011, Fuzzy Optim. Decis. Mak..

[45]  Jerry M. Mendel,et al.  Uncertainty Measures for General Type-2 Fuzzy Sets , 2009, SMC.