Knowledge Entropy and Feature Selection in Incomplete Decision Systems

In this paper, concepts of knowledge entropy and knowledge entropy-based uncertainty measures are given in incomplete information systems and decision systems, and some important properties of them are investigated. From these properties, it can be shown that these measures provide important approaches to measure the uncertainty ability of different knowledge in incomplete decision systems. Then the relationships among these knowledge entropies proposed are discussed as well. A new definition of reduct is proposed and a heuristic algorithm with low computational complexity is constructed to improve computational efficiency of feature selection in incomplete decision systems. Experimental results demonstrate that our algorithm can provide an efficient solution to find a minimal subset of the features from incomplete data sets.

[1]  Xiangyang Wang,et al.  Feature selection based on rough sets and particle swarm optimization , 2007, Pattern Recognit. Lett..

[2]  Qinghua Hu,et al.  Fuzzy probabilistic approximation spaces and their information measures , 2006, IEEE Transactions on Fuzzy Systems.

[3]  Jiye Liang,et al.  Information granules and entropy theory in information systems , 2008, Science in China Series F: Information Sciences.

[4]  Xu Weihua,et al.  Knowledge granulation, knowledge entropy and knowledge uncertainty measure in ordered information systems , 2009 .

[5]  Lin Sun,et al.  Feature selection using rough entropy-based uncertainty measures in incomplete decision systems , 2012, Knowl. Based Syst..

[6]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[7]  Xu Zhang,et al.  A Quick Attribute Reduction Algorithm with Complexity of max(O(｜C｜｜U｜),O(｜C｜~2｜U/C｜)) , 2006 .

[8]  Jiye Liang,et al.  Combination Entropy and Combination Granulation in Rough Set Theory , 2008, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[9]  Guoyin Wang,et al.  Rough reduction in algebra view and information view , 2003, Int. J. Intell. Syst..

[10]  Lin Sun,et al.  Granularity Partition-based Feature Selection and Its Application in Decision Systems , 2012 .

[11]  Janusz A. Starzyk,et al.  Reduct Generation in Information Systems , 2002 .

[12]  Marzena Kryszkiewicz,et al.  Rough Set Approach to Incomplete Information Systems , 1998, Inf. Sci..

[13]  Wen-Xiu Zhang,et al.  Knowledge granulation, knowledge entropy and knowledge uncertainty measure in ordered information systems , 2009, Appl. Soft Comput..

[14]  Zhongzhi Shi,et al.  A fast approach to attribute reduction in incomplete decision systems with tolerance relation-based rough sets , 2009, Inf. Sci..