Fuzzy rough sets and fuzzy rough neural networks for feature selection: A review

Feature selection aims to select a feature subset from an original feature set based on a certain evaluation criterion. Since feature selection can achieve efficient feature reduction, it has become a key method for data preprocessing in many data mining tasks. Recently, many feature selection strategies have been developed since in most cases it is infeasible to obtain an optimal/reduced feature subset by using exhaustive search. Among these strategies, fuzzy rough set theory has proved to be an ideal candidate for dealing with uncertain information. This article provides a comprehensive review on the fuzzy rough set theory and two fuzzy rough set theory based feature selection methods, that is, fuzzy rough set based feature selection methods and fuzzy rough neural network based feature selection methods. We review the publications related to the fuzzy rough theory and its applications in feature selection. In addition, the challenges in the two types of feature selection methods are also discussed.

[1]  Daren Yu,et al.  Entropies Of Fuzzy Indiscernibility Relation And Its Operations , 2005, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[2]  Xizhao Wang,et al.  Attributes Reduction Using Fuzzy Rough Sets , 2008, IEEE Transactions on Fuzzy Systems.

[3]  Xizhao Wang,et al.  On the generalization of fuzzy rough sets , 2005, IEEE Transactions on Fuzzy Systems.

[4]  Qinghua Hu,et al.  Fuzzy probabilistic approximation spaces and their information measures , 2006, IEEE Transactions on Fuzzy Systems.

[5]  Chris Cornelis,et al.  Ordered Weighted Average Based Fuzzy Rough Sets , 2010, RSKT.

[6]  Witold Pedrycz,et al.  Gaussian kernel based fuzzy rough sets: Model, uncertainty measures and applications , 2010, Int. J. Approx. Reason..

[7]  Yaojin Lin,et al.  A graph approach for fuzzy-rough feature selection , 2020, Fuzzy Sets Syst..

[8]  Ruili Wang,et al.  Joint self-representation and subspace learning for unsupervised feature selection , 2017, World Wide Web.

[9]  Nikola Bogunovic,et al.  A review of feature selection methods with applications , 2015, 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[10]  Jesús Manuel Fernández Salido,et al.  Rough set analysis of a general type of fuzzy data using transitive aggregations of fuzzy similarity relations , 2003, Fuzzy Sets Syst..

[11]  Sushmita Mitra,et al.  Feature Selection Using Radial Basis Function Networks , 1999, Neural Computing & Applications.

[12]  Sankar K. Pal,et al.  Fuzzy rough granular self-organizing map and fuzzy rough entropy , 2012, Theor. Comput. Sci..

[13]  Chris Cornelis,et al.  Applications of Fuzzy Rough Set Theory in Machine Learning: a Survey , 2015, Fundam. Informaticae.

[14]  Zhang Dong-bo Fuzzy-Rough Membership Function Neural Network Suitable for Pattern Classification , 2007 .

[15]  Ming-Wen Shao,et al.  Set approximations in fuzzy formal concept analysis , 2007, Fuzzy Sets Syst..

[16]  Rajen B. Bhatt,et al.  On the compact computational domain of fuzzy-rough sets , 2005, Pattern Recognit. Lett..

[17]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[18]  Qiang Shen,et al.  Selecting informative features with fuzzy-rough sets and its application for complex systems monitoring , 2004, Pattern Recognit..

[19]  Zhoujun Li,et al.  A novel variable precision (θ, σ)-fuzzy rough set model based on fuzzy granules , 2014, Fuzzy Sets Syst..

[20]  Qiang Shen,et al.  A Distance Measure Approach to Exploring the Rough Set Boundary Region for Attribute Reduction , 2010, IEEE Transactions on Knowledge and Data Engineering.

[21]  Qiang Shen,et al.  Fuzzy-rough sets for descriptive dimensionality reduction , 2002, 2002 IEEE World Congress on Computational Intelligence. 2002 IEEE International Conference on Fuzzy Systems. FUZZ-IEEE'02. Proceedings (Cat. No.02CH37291).

[22]  Qiang Shen,et al.  New Approaches to Fuzzy-Rough Feature Selection , 2009, IEEE Transactions on Fuzzy Systems.

[23]  Wei Wei,et al.  Consistency-preserving attribute reduction in fuzzy rough set framework , 2013, Int. J. Mach. Learn. Cybern..

[24]  Yang Li,et al.  Feature selection for transient stability assessment based on kernelized fuzzy rough sets and memetic algorithm , 2018, 1808.08790.

[25]  Jiye Liang,et al.  Fuzzy-rough feature selection accelerator , 2015, Fuzzy Sets Syst..

[26]  Qinghua Hu,et al.  Soft fuzzy rough sets for robust feature evaluation and selection , 2010, Inf. Sci..

[27]  Qiang Shen,et al.  Fuzzy Entropy-assisted Fuzzy-Rough Feature Selection , 2006, 2006 IEEE International Conference on Fuzzy Systems.

[28]  LarrañagaPedro,et al.  A review of feature selection techniques in bioinformatics , 2007 .

[29]  Qinghua Hu,et al.  Robust fuzzy rough classifiers , 2011, Fuzzy Sets Syst..

[30]  Mital Doshi,et al.  CORRELATION BASED FEATURE SELECTION (CFS) TECHNIQUE TO PREDICT STUDENT PERFROMANCE , 2014 .

[31]  Rajen B. Bhatt,et al.  FRCT: fuzzy-rough classification trees , 2007, Pattern Analysis and Applications.

[32]  Yuwen Li,et al.  Attribute reduction for multi-label learning with fuzzy rough set , 2018, Knowl. Based Syst..

[33]  Qinghua Hu,et al.  On Robust Fuzzy Rough Set Models , 2012, IEEE Transactions on Fuzzy Systems.

[34]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[35]  Sankar K. Pal,et al.  Fuzzy Rough Granular Self Organizing Map , 2011, RSKT.

[36]  Xun Wang,et al.  Review on mining data from multiple data sources , 2018, Pattern Recognit. Lett..

[37]  Qinghua Hu,et al.  Parameterized attribute reduction with Gaussian kernel based fuzzy rough sets , 2011, Inf. Sci..

[38]  Huan Liu,et al.  A Probabilistic Approach to Feature Selection - A Filter Solution , 1996, ICML.

[39]  Kok Wai Wong,et al.  Feature decreasing methods using fuzzy rough set based on mutual information , 2013, 2013 IEEE 8th Conference on Industrial Electronics and Applications (ICIEA).

[40]  Yaonan Wang,et al.  Remote sensing images classification using fuzzy-rough neural network , 2010, BIC-TA.

[41]  Xiao Zhang,et al.  A fuzzy rough set-based feature selection method using representative instances , 2018, Knowl. Based Syst..

[42]  Zhi-Li Zhang,et al.  Fuzzy rough neural network and its application to feature selection , 2011, The Fourth International Workshop on Advanced Computational Intelligence.

[43]  D. Dubois,et al.  ROUGH FUZZY SETS AND FUZZY ROUGH SETS , 1990 .

[44]  Qinghua Hu,et al.  Fuzzy preference based rough sets , 2010, Inf. Sci..

[45]  Degang Chen,et al.  Attribute Reduction for Heterogeneous Data Based on the Combination of Classical and Fuzzy Rough Set Models , 2014, IEEE Transactions on Fuzzy Systems.

[46]  Witold Pedrycz,et al.  Data-Distribution-Aware Fuzzy Rough Set Model and its Application to Robust Classification , 2016, IEEE Transactions on Cybernetics.

[47]  Wei Zheng,et al.  Self-representation and PCA embedding for unsupervised feature selection , 2017, World Wide Web.

[48]  Aboul Ella Hassanien,et al.  Fuzzy rough sets hybrid scheme for breast cancer detection , 2007, Image Vis. Comput..

[49]  Ansheng Deng,et al.  Associated multi-label fuzzy-rough feature selection , 2017, 2017 Joint 17th World Congress of International Fuzzy Systems Association and 9th International Conference on Soft Computing and Intelligent Systems (IFSA-SCIS).

[50]  Rajen B. Bhatt,et al.  On fuzzy-rough sets approach to feature selection , 2005, Pattern Recognit. Lett..

[51]  A. Sunny Kuriakose,et al.  A novel feature selection method using fuzzy rough sets , 2018, Comput. Ind..

[52]  Partha Garai,et al.  Fuzzy–Rough Simultaneous Attribute Selection and Feature Extraction Algorithm , 2013, IEEE Transactions on Cybernetics.

[53]  Guoyin Wang,et al.  A Comparative Study of Algebra Viewpoint and Information Viewpoint in Attribute Reduction , 2005, Fundam. Informaticae.

[54]  Francisco Herrera,et al.  Implementing algorithms of rough set theory and fuzzy rough set theory in the R package "RoughSets" , 2014, Inf. Sci..

[55]  S. Ramakrishnan,et al.  Attribute selection using fuzzy roughset based customized similarity measure for lung cancer microarray gene expression data , 2018, Future Computing and Informatics Journal.

[56]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[57]  Yi-Fan Wang,et al.  Mining stock price using fuzzy rough set system , 2003, Expert Syst. Appl..

[58]  Giancarlo Fortino,et al.  Intelligent temporal classification and fuzzy rough set-based feature selection algorithm for intrusion detection system in WSNs , 2019, Inf. Sci..

[59]  Francisco Herrera,et al.  Dynamic affinity-based classification of multi-class imbalanced data with one-versus-one decomposition: a fuzzy rough set approach , 2018, Knowledge and Information Systems.

[60]  Jianping Yin,et al.  Relational recurrent neural networks for polyphonic sound event detection , 2019, Multimedia Tools and Applications.

[61]  Nehad N. Morsi,et al.  Axiomatics for fuzzy rough sets , 1998, Fuzzy Sets Syst..

[62]  Manish Aggarwal,et al.  Probabilistic Variable Precision Fuzzy Rough Sets , 2016, IEEE Transactions on Fuzzy Systems.

[63]  Pawan Lingras,et al.  Comparison of Neofuzzy and Rough Neural Networks , 1998, Inf. Sci..

[64]  Kok Wai Wong,et al.  Utilising fuzzy rough set based on mutual information decreasing method for feature reduction in an image retrieval system , 2014 .

[65]  Xiao Zhang,et al.  Feature selection in mixed data: A method using a novel fuzzy rough set-based information entropy , 2016, Pattern Recognit..

[66]  Ming-Wen Shao,et al.  Feature subset selection based on fuzzy neighborhood rough sets , 2016, Knowl. Based Syst..

[67]  Qinghua Hu,et al.  A Fitting Model for Feature Selection With Fuzzy Rough Sets , 2017, IEEE Transactions on Fuzzy Systems.

[68]  Hui Wang,et al.  Fuzzy rough set based incremental attribute reduction from dynamic data with sample arriving , 2017, Fuzzy Sets Syst..

[69]  Dun Liu,et al.  A fuzzy rough set approach for incremental feature selection on hybrid information systems , 2015, Fuzzy Sets Syst..

[70]  J. Clausen,et al.  Branch and Bound Algorithms-Principles and Examples , 2003 .

[71]  Theresa Beaubouef,et al.  Rough Sets , 2019, Lecture Notes in Computer Science.

[72]  Aiguo Song,et al.  Regularization feature selection projection twin support vector machine via exterior penalty , 2017, Neural Computing and Applications.

[73]  Huan Liu,et al.  Neural-network feature selector , 1997, IEEE Trans. Neural Networks.

[74]  Qiang Shen,et al.  Centre for Intelligent Systems and Their Applications Fuzzy Rough Attribute Reduction with Application to Web Categorization Fuzzy Rough Attribute Reduction with Application to Web Categorization Fuzzy Sets and Systems ( ) – Fuzzy–rough Attribute Reduction with Application to Web Categorization , 2022 .

[75]  Sam Kwong,et al.  Geometrical interpretation and applications of membership functions with fuzzy rough sets , 2012, Fuzzy Sets Syst..

[76]  Simon C. K. Shiu,et al.  Combining feature reduction and case selection in building CBR classifiers , 2006, IEEE Transactions on Knowledge and Data Engineering.

[77]  Pradipta Maji,et al.  Fuzzy–Rough Supervised Attribute Clustering Algorithm and Classification of Microarray Data , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[78]  Wei-Zhi Wu,et al.  Generalized fuzzy rough sets , 2003, Inf. Sci..

[79]  Wojciech Ziarko,et al.  Variable Precision Rough Set Model , 1993, J. Comput. Syst. Sci..

[80]  Sankar K. Pal,et al.  Unsupervised feature evaluation: a neuro-fuzzy approach , 2000, IEEE Trans. Neural Networks Learn. Syst..

[81]  J. Recasens,et al.  UPPER AND LOWER APPROXIMATIONS OF FUZZY SETS , 2000 .

[82]  Dominik Slezak,et al.  Approximate Entropy Reducts , 2002, Fundam. Informaticae.

[84]  Qinghua Hu,et al.  A Novel Algorithm for Finding Reducts With Fuzzy Rough Sets , 2012, IEEE Transactions on Fuzzy Systems.

[85]  Witold Pedrycz,et al.  Kernelized Fuzzy Rough Sets and Their Applications , 2011, IEEE Transactions on Knowledge and Data Engineering.

[86]  W. Pedrycz Why triangular membership functions , 1994 .

[87]  Antanas Verikas,et al.  Feature selection with neural networks , 2002, Pattern Recognit. Lett..

[88]  Qinghua Hu,et al.  Hybrid attribute reduction based on a novel fuzzy-rough model and information granulation , 2007, Pattern Recognit..

[89]  Chao Liu,et al.  New definition of dependency in fuzzy rough feature selection , 2011, 2011 International Conference on System science, Engineering design and Manufacturing informatization.

[90]  Robert LIN,et al.  NOTE ON FUZZY SETS , 2014 .

[91]  Ruili Wang,et al.  Durable relationship prediction and description using a large dynamic graph , 2017, World Wide Web.

[92]  Ruili Wang,et al.  Feature selection for least squares projection twin support vector machine , 2014, Neurocomputing.

[93]  Qinghua Hu,et al.  Information-preserving hybrid data reduction based on fuzzy-rough techniques , 2006, Pattern Recognit. Lett..

[94]  Qiang Shen,et al.  Fuzzy-Rough Sets Assisted Attribute Selection , 2007, IEEE Transactions on Fuzzy Systems.

[95]  Manish Sarkar,et al.  Fuzzy-rough neural networks for vowel classification , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[96]  Ruili Wang,et al.  An enhanced fuzzy linear regression model with more flexible spreads , 2009, Fuzzy Sets Syst..

[97]  Anna Maria Radzikowska,et al.  A comparative study of fuzzy rough sets , 2002, Fuzzy Sets Syst..

[98]  Witold Pedrycz,et al.  Large-Scale Multimodality Attribute Reduction With Multi-Kernel Fuzzy Rough Sets , 2018, IEEE Transactions on Fuzzy Systems.

[99]  V. Valli Kumari,et al.  Feature Selection Using Relative Fuzzy Entropy and Ant Colony Optimization Applied to Real-time Intrusion Detection System , 2016 .

[100]  Xizhao Wang,et al.  Learning fuzzy rules from fuzzy samples based on rough set technique , 2007, Inf. Sci..

[101]  Wei-Zhi Wu,et al.  Maximal-Discernibility-Pair-Based Approach to Attribute Reduction in Fuzzy Rough Sets , 2018, IEEE Transactions on Fuzzy Systems.

[102]  Alicja Mieszkowicz-Rolka,et al.  Fuzzy rough approximations of process data , 2008, Int. J. Approx. Reason..

[103]  Wei Wei,et al.  A comparative study of rough sets for hybrid data , 2012, Inf. Sci..

[104]  T. Hong,et al.  Learning a coverage set of maximally general fuzzy rules by rough sets , 2000 .

[105]  Alicja Mieszkowicz-Rolka,et al.  Variable Precision Fuzzy Rough Sets , 2004, Trans. Rough Sets.

[106]  Soumitra Dutta,et al.  Fuzzy rough granular neural networks, fuzzy granules, and classification , 2011, Theor. Comput. Sci..

[107]  Degang Chen,et al.  The Model of Fuzzy Variable Precision Rough Sets , 2009, IEEE Transactions on Fuzzy Systems.

[108]  Juan M. Corchado,et al.  Reducing the Memory Size of a Fuzzy Case-Based Reasoning System Applying Rough Set Techniques , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[109]  L. Kuncheva Fuzzy rough sets: application to feature selection , 1992 .

[110]  Qiang Shen,et al.  Semantics-preserving dimensionality reduction: rough and fuzzy-rough-based approaches , 2004, IEEE Transactions on Knowledge and Data Engineering.

[111]  Wen-Xiu Zhang,et al.  An axiomatic characterization of a fuzzy generalization of rough sets , 2004, Inf. Sci..

[112]  Pawan Lingras,et al.  Fuzzy-rough and rough-fuzzy serial combinations in neurocomputing , 2001, Neurocomputing.

[113]  Wang Yaonan,et al.  Fuzzy-rough Neural Network and Its Application to Vowel Recognition , 2006 .

[114]  Manfred M. Fischer,et al.  A Rough Set Approach for the Discovery of Classification Rules in Interval-Valued Information Systems , 2008, Int. J. Approx. Reason..

[115]  Ming Zong,et al.  Robust multi-view continuous subspace clustering , 2018, Pattern Recognit. Lett..

[116]  Mona Gamal Gafar,et al.  Medical Diagnostic System Basing Fuzzy Rough Neural-Computing for Breast Cancer , 2016, AISI.

[117]  Sankar K. Pal,et al.  Fuzzy rough sets, and a granular neural network for unsupervised feature selection , 2013, Neural Networks.

[118]  Xizhao Wang,et al.  Incremental Perspective for Feature Selection Based on Fuzzy Rough Sets , 2018, IEEE Transactions on Fuzzy Systems.

[119]  Thomas G. Dietterich,et al.  Learning with Many Irrelevant Features , 1991, AAAI.

[120]  Sankar K. Pal,et al.  A Novel Fuzzy Rough Granular Neural Network for Classification , 2011, Int. J. Comput. Intell. Syst..