Semantics-preserving dimensionality reduction: rough and fuzzy-rough-based approaches

Semantics-preserving dimensionality reduction refers to the problem of selecting those input features that are most predictive of a given outcome; a problem encountered in many areas such as machine learning, pattern recognition, and signal processing. This has found successful application in tasks that involve data sets containing huge numbers of features (in the order of tens of thousands), which would be impossible to process further. Recent examples include text processing and Web content classification. One of the many successful applications of rough set theory has been to this feature selection area. This paper reviews those techniques that preserve the underlying semantics of the data, using crisp and fuzzy rough set-based methodologies. Several approaches to feature selection based on rough set theory are experimentally compared. Additionally, a new area in feature selection, feature grouping, is highlighted and a rough set-based feature grouping technique is detailed.

[1]  Qiang Shen,et al.  Finding Rough Set Reducts with Ant Colony Optimization , 2003 .

[2]  Huan Liu,et al.  Neural-network feature selector , 1997, IEEE Trans. Neural Networks.

[3]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[4]  Krzysztof Krawiec,et al.  ROUGH SET REDUCTION OF ATTRIBUTES AND THEIR DOMAINS FOR NEURAL NETWORKS , 1995, Comput. Intell..

[5]  Dominik Slezak,et al.  Normalized Decision Functions and Measures for Inconsistent Decision Tables Analysis , 2000, Fundam. Informaticae.

[6]  Qiang Shen,et al.  Using Fuzzy Dependency-Guided Attribute Grouping in Feature Selection , 2003, RSFDGrC.

[7]  Andrzej Skowron,et al.  Boolean Reasoning for Feature Extraction Problems , 1997, ISMIS.

[8]  Alexis Tsoukiàs,et al.  Valued Tolerance and Decision Rules , 2000, Rough Sets and Current Trends in Computing.

[9]  Z. Pawlak,et al.  Rough membership functions , 1994 .

[10]  Andrzej Skowron,et al.  Rough-Fuzzy Hybridization: A New Trend in Decision Making , 1999 .

[11]  Yiyu Yao,et al.  Rough Sets and Current Trends in Computing , 2001, Lecture Notes in Computer Science.

[12]  Dominik Slezak,et al.  Approximate Reducts in Decision Tables , 1996 .

[13]  Yiyu Yao,et al.  Constructive and Algebraic Methods of the Theory of Rough Sets , 1998, Inf. Sci..

[14]  Maciej Modrzejewski,et al.  Feature Selection Using Rough Sets Theory , 1993, ECML.

[15]  Andrzej Skowron,et al.  Rough set methods in feature selection and recognition , 2003, Pattern Recognit. Lett..

[16]  Sadaaki Miyamoto,et al.  Rough Sets and Current Trends in Computing , 2012, Lecture Notes in Computer Science.

[17]  Alan J. Miller,et al.  Subset Selection in Regression , 1991 .

[18]  Salvatore Greco,et al.  Fuzzy Similarity Relation as a Basis for Rough Approximations , 1998, Rough Sets and Current Trends in Computing.

[19]  Yiyu Yao,et al.  A Comparative Study of Fuzzy Sets and Rough Sets , 1998 .

[20]  Qiang Shen,et al.  Selecting informative features with fuzzy-rough sets and its application for complex systems monitoring , 2004, Pattern Recognit..

[21]  Wojciech Ziarko,et al.  Variable Precision Rough Set Model , 1993, J. Comput. Syst. Sci..

[22]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[23]  Ivo Düntsch,et al.  Rough set data analysis: A road to non-invasive knowledge discovery , 2000 .

[24]  Jan G. Bazan.  Chapter 17 a Comparison of Dynamic and Non{dynamic Rough Set Methods for Extracting Laws from Decision Tables , 1998 .

[25]  Vijay V. Raghavan,et al.  The Status of Research on Rough Sets for Knowledge Discovery in Databases , 1998 .

[26]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[27]  Qiang Shen,et al.  Rough set-aided keyword reduction for text categorization , 2001, Appl. Artif. Intell..

[28]  Thomas G. Dietterich,et al.  Learning with Many Irrelevant Features , 1991, AAAI.

[29]  Malcolm J. Beynon.  An Investigation of beta-Reduct Selection within the Variable Precision Rough Sets Model , 2000, Rough Sets and Current Trends in Computing.

[30]  A. Atkinson.  Subset Selection in Regression , 1992 .

[31]  Andrzej Skowron,et al.  Dynamic Reducts as a Tool for Extracting Laws from Decisions Tables , 1994, ISMIS.

[32]  Andrzej Skowron,et al.  New Directions in Rough Sets, Data Mining, and Granular-Soft Computing , 1999, Lecture Notes in Computer Science.

[33]  C.J.H. Mann,et al.  Handbook of Data Mining and Knowledge Discovery , 2004 .

[34]  B. Raman,et al.  Instance Based Filter for Feature Selection , 2002 .

[35]  Z. Pawlak,et al.  Rough sets perspective on data and knowledge , 2002 .

[36]  P. Langley.  Selection of Relevant Features in Machine Learning , 1994 .

[37]  Jeffrey C. Schlimmer,et al.  Efficiently Inducing Determinations: A Complete and Systematic Search Algorithm that Uses Optimal Pruning , 1993, ICML.

[38]  Janusz Zalewski,et al.  Rough sets: Theoretical aspects of reasoning about data , 1996 .

[39]  Ning Zhong,et al.  Using Rough Sets with Heuristics for Feature Selection , 1999, RSFDGrC.

[40]  Andrzej Skowron,et al.  The Discernibility Matrices and Functions in Information Systems , 1992, Intelligent Decision Support.

[41]  Maciej Wygralak.  Rough sets and fuzzy sets—some remarks on interrelations , 1989 .

[42]  Larry A. Rendell,et al.  The Feature Selection Problem: Traditional Methods and a New Algorithm , 1992, AAAI.

[43]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[44]  Andrzej Skowron,et al.  Tolerance Approximation Spaces , 1996, Fundam. Informaticae.

[45]  Qiang Shen,et al.  Centre for Intelligent Systems and Their Applications Fuzzy Rough Attribute Reduction with Application to Web Categorization Fuzzy Rough Attribute Reduction with Application to Web Categorization Fuzzy Sets and Systems ( ) – Fuzzy–rough Attribute Reduction with Application to Web Categorization , 2022 .

[46]  Andrzej Skowron,et al.  Rough Sets: A Tutorial , 1998 .

[47]  Erol Gelenbe,et al.  Computer and Information Sciences - 3 , 1989 .

[48]  Qiang Shen,et al.  Rough Feature Selection for Neural Network Based Image Classification , 2002, Int. J. Image Graph..

[49]  D. Vanderpooten.  Similarity Relation as a Basis for Rough Approximations , 1995 .

[50]  J. Kacprzyk,et al.  Advances in the Dempster-Shafer theory of evidence , 1994 .

[51]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[52]  H. Thiele.  Fuzzy Rough Sets versus Rough Fuzzy Sets — An Interpretation and a Comparative Study using Concepts of Modal Logics , 1998 .

[53]  Grzegorz Drwal.  Rough and Fuzzy-Rough Classification Methods Implemented in RClass System , 2000, Rough Sets and Current Trends in Computing.

[54]  Malcolm J. Beynon,et al.  Reducts within the variable precision rough sets model: A further investigation , 2001, Eur. J. Oper. Res..

[55]  Marzena Kryszkiewicz.  Maintenance of Reducts in the Variable Precision Rough Set Model , 1997 .

[56]  S. Tsumoto,et al.  Rough set methods and applications: new developments in knowledge discovery in information systems , 2000 .

[57]  Ivo Düntsch,et al.  Rough Set Data Analysis , 2000 .

[58]  Ning Zhong,et al.  Using Rough Sets with Heuristics for Feature Selection , 1999, Journal of Intelligent Information Systems.

[59]  Andrzej Skowron,et al.  From the Rough Set Theory to the Evidence Theory , 1991 .

[60]  U. Höhle.  Quotients with respect to similarity relations , 1988 .

[61]  Thomas G. Dietterich,et al.  Learning Boolean Concepts in the Presence of Many Irrelevant Features , 1994, Artif. Intell..

[62]  Didier Dubois,et al.  Putting Rough Sets and Fuzzy Sets Together , 1992, Intelligent Decision Support.

[63]  Qiang Shen,et al.  A rough-fuzzy approach for generating classification rules , 2002, Pattern Recognit..

[64]  Hiroshi Motoda,et al.  Feature Extraction, Construction and Selection: A Data Mining Perspective , 1998 .