Feature selection via neighborhood multi-granulation fusion

Feature selection is an important data preprocessing technique, and has been widely studied in data mining, machine learning, and granular computing. However, very little research has considered a multi-granulation perspective. In this paper, we present a new feature selection method that selects distinguishing features by fusing neighborhood multi-granulation. We first use neighborhood rough sets as an effective granular computing tool, and analyze the influence of the granularity of neighborhood information. Then, we obtain all feature rank lists based on the significance of features in different granularities. Finally, we obtain a new feature selection algorithm by fusing all individual feature rank lists. Experimental results show that the proposed method can effectively select a discriminative feature subset, and performs as well as or better than other popular feature selection algorithms in terms of classification performance.

[1]  Witold Pedrycz,et al.  Feature analysis through information granulation and fuzzy sets , 2002, Pattern Recognit..

[2]  Lotfi A. Zadeh,et al.  Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic , 1997, Fuzzy Sets Syst..

[3]  Yiyu Yao,et al.  Three-way decisions with probabilistic rough sets , 2010, Inf. Sci..

[4]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[5]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[6]  Qinghua Hu,et al.  Feature selection with test cost constraint , 2012, ArXiv.

[7]  Qinghua Hu,et al.  Adaptive neighborhood granularity selection and combination based on margin distribution optimization , 2013, Inf. Sci..

[8]  Qinghua Hu,et al.  Neighborhood classifiers , 2008, Expert Syst. Appl..

[9]  Yiyu Yao,et al.  MGRS: A multi-granulation rough set , 2010, Inf. Sci..

[10]  Dirk P. Kroese,et al.  The Cross Entropy Method: A Unified Approach To Combinatorial Optimization, Monte-carlo Simulation (Information Science and Statistics) , 2004 .

[11]  Dirk P. Kroese,et al.  Cross‐Entropy Method , 2011 .

[12]  Qiang Shen,et al.  Finding Rough Set Reducts with SAT , 2005, RSFDGrC.

[13]  Witold Pedrycz,et al.  Granular Computing: Perspectives and Challenges , 2013, IEEE Transactions on Cybernetics.

[14]  Jiye Liang,et al.  International Journal of Approximate Reasoning an Efficient Rough Feature Selection Algorithm with a Multi-granulation View , 2022 .

[15]  Qinghua Hu,et al.  Rule learning for classification based on neighborhood covering reduction , 2011, Inf. Sci..

[16]  Duoqian Miao,et al.  Two basic double-quantitative rough set models of precision and grade and their investigation using granular computing , 2013, Int. J. Approx. Reason..

[17]  Gavin Brown,et al.  Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection , 2012, J. Mach. Learn. Res..

[18]  Xiaomei Li,et al.  Mining stable patterns in multiple correlated databases , 2013, Decis. Support Syst..

[19]  Jianhua Dai,et al.  Attribute selection based on information gain ratio in fuzzy rough set theory with application to tumor classification , 2013, Appl. Soft Comput..

[20]  Yiyu Yao,et al.  Neighborhood systems and approximate retrieval , 2006, Inf. Sci..

[21]  Witold Pedrycz,et al.  Selecting Discrete and Continuous Features Based on Neighborhood Decision Error Minimization , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  Qinghua Hu,et al.  Mixed feature selection based on granulation and approximation , 2008, Knowl. Based Syst..

[23]  Ronald Fagin,et al.  Comparing top k lists , 2003, SODA '03.

[24]  Witold Pedrycz,et al.  Granular Computing: Analysis and Design of Intelligent Systems , 2013 .

[25]  Jiye Liang,et al.  Incomplete Multigranulation Rough Set , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[26]  Yee Leung,et al.  Optimal scale selection for multi-scale decision tables , 2013, Int. J. Approx. Reason..

[27]  Huan Liu,et al.  Consistency-based search in feature selection , 2003, Artif. Intell..

[28]  Xindong Wu,et al.  Quality of information-based source assessment and selection , 2014, Neurocomputing.

[29]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Wei-Zhi Wu,et al.  Neighborhood operator systems and approximations , 2002, Inf. Sci..

[31]  Yiyu Yao,et al.  Relational Interpretations of Neigborhood Operators and Rough Set Approximation Operators , 1998, Inf. Sci..

[32]  Wu Shunxiang Rough set theory in interval and set-valued information systems , 2011 .

[33]  Wang Guo,et al.  Decision Table Reduction based on Conditional Information Entropy , 2002 .

[34]  Jiye Liang,et al.  Multigranulation rough sets: From partition to covering , 2013, Inf. Sci..

[35]  Dun Liu,et al.  A Multiple-category Classification Approach with Decision-theoretic Rough Sets , 2012, Fundam. Informaticae.

[36]  T. Lin Granulation and nearest neighborhoods: rough set approach , 2001 .

[37]  Fei-Yue Wang,et al.  Reduction and axiomization of covering generalized rough sets , 2003, Inf. Sci..

[38]  Qinghua Hu,et al.  Feature Selection for Monotonic Classification , 2012, IEEE Transactions on Fuzzy Systems.

[39]  Yiyu Yao,et al.  Probabilistic rough set approximations , 2008, Int. J. Approx. Reason..

[40]  Qinghua Hu,et al.  Neighborhood rough set based heterogeneous feature subset selection , 2008, Inf. Sci..

[41]  Jingcheng Wang,et al.  Neighborhood effective information ratio for hybrid feature subset evaluation and selection , 2013, Neurocomputing.

[42]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[43]  Xindong Wu,et al.  Ensemble learning from multiple information sources via label propagation and consensus , 2013, Applied Intelligence.

[44]  Zhenmin Tang,et al.  Minimum cost attribute reduction in decision-theoretic rough set models , 2013, Inf. Sci..

[45]  W. Li,et al.  Hybrid approaches to attribute reduction based on indiscernibility and discernibility relation , 2011, Int. J. Approx. Reason..

[46]  Huan Liu,et al.  Feature selection for classification: A review , 2014 .

[47]  Zhao Zhang,et al.  Locality preserving multimodal discriminative learning for supervised feature selection , 2011, Knowledge and Information Systems.

[48]  Yuhua Qian,et al.  NMGRS: Neighborhood-based multigranulation rough sets , 2012, Int. J. Approx. Reason..

[49]  Dirk P. Kroese,et al.  The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning , 2004 .

[50]  Y. Yao Information granulation and rough set approximation , 2001 .