Parameterized attribute reduction with Gaussian kernel based fuzzy rough sets

Fuzzy rough sets are considered as an effective tool to deal with uncertainty in data analysis, and fuzzy similarity relations are used in fuzzy rough sets to calculate similarity between objects. On the other hand in kernel tricks, a kernel maps data into a higher dimensional feature space where the resulting structure of the learning task is linearly separable, while the kernel is the inner product of this feature space and can also be viewed as a similarity function. It has been reported there is an overlap between family of kernels and collection of fuzzy similarity relations. This fact motivates the idea in this paper to use some kernels as fuzzy similarity relations and develop kernel based fuzzy rough sets. First, we consider Gaussian kernel and propose Gaussian kernel based fuzzy rough sets. Second we introduce parameterized attribute reduction with the derived model of fuzzy rough sets. Structures of attribute reduction are investigated and an algorithm with discernibility matrix to find all reducts is developed. Finally, a heuristic algorithm is designed to compute reducts with Gaussian kernel fuzzy rough sets. Several experiments are provided to demonstrate the effectiveness of the idea.

[1]  Eric C. C. Tsang,et al.  On fuzzy approximation operators in attribute reduction with fuzzy rough sets , 2008, Inf. Sci..

[2]  Gabriele Steidl,et al.  Combined SVM-Based Feature Selection and Classification , 2005, Machine Learning.

[3]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[4]  Qiang Shen,et al.  Fuzzy-Rough Sets Assisted Attribute Selection , 2007, IEEE Transactions on Fuzzy Systems.

[5]  Anna Maria Radzikowska,et al.  A comparative study of fuzzy rough sets , 2002, Fuzzy Sets Syst..

[6]  Qinghua Hu,et al.  Information-preserving hybrid data reduction based on fuzzy-rough techniques , 2006, Pattern Recognit. Lett..

[7]  Qinghua Hu,et al.  Hybrid attribute reduction based on a novel fuzzy-rough model and information granulation , 2007, Pattern Recognit..

[8]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[9]  Radko Mesiar,et al.  Triangular Norms , 2000, Trends in Logic.

[10]  Xizhao Wang,et al.  On the generalization of fuzzy rough sets , 2005, IEEE Transactions on Fuzzy Systems.

[11]  Yi Liu,et al.  FS_SFS: A novel feature selection method for support vector machines , 2006, Pattern Recognit..

[12]  Qinghua Hu,et al.  Uncertainty measures for fuzzy relations and their applications , 2007, Appl. Soft Comput..

[13]  Jesús Manuel Fernández Salido,et al.  Rough set analysis of a general type of fuzzy data using transitive aggregations of fuzzy similarity relations , 2003, Fuzzy Sets Syst..

[14]  David G. Stork,et al.  Pattern Classification , 1973 .

[15]  Yee Leung,et al.  Generalized fuzzy rough sets determined by a triangular norm , 2008, Inf. Sci..

[16]  Francesc Esteva,et al.  Review of Triangular norms by E. P. Klement, R. Mesiar and E. Pap. Kluwer Academic Publishers , 2003 .

[17]  Qiang Shen,et al.  New Approaches to Fuzzy-Rough Feature Selection , 2009, IEEE Transactions on Fuzzy Systems.

[18]  Robert Tibshirani,et al.  1-norm Support Vector Machines , 2003, NIPS.

[19]  Mark A. Hall,et al.  Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning , 1999, ICML.

[20]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[21]  Xizhao Wang,et al.  Attributes Reduction Using Fuzzy Rough Sets , 2008, IEEE Transactions on Fuzzy Systems.

[22]  Jiun-Hung Chen,et al.  Fuzzy kernel perceptron , 2002, IEEE Trans. Neural Networks.

[23]  Wei-Zhi Wu,et al.  Generalized fuzzy rough sets , 2003, Inf. Sci..

[24]  Wei-Zhi Wu,et al.  Constructive and axiomatic approaches of fuzzy approximation operators , 2004, Inf. Sci..

[25]  Victor L. Brailovsky,et al.  On domain knowledge and feature selection using a support vector machine , 1999, Pattern Recognit. Lett..

[26]  Qiang Shen,et al.  Centre for Intelligent Systems and Their Applications Fuzzy Rough Attribute Reduction with Application to Web Categorization Fuzzy Rough Attribute Reduction with Application to Web Categorization Fuzzy Sets and Systems ( ) – Fuzzy–rough Attribute Reduction with Application to Web Categorization , 2022 .

[27]  Witold Pedrycz,et al.  Gaussian kernel based fuzzy rough sets: Model, uncertainty measures and applications , 2010, Int. J. Approx. Reason..

[28]  Cheng-Lung Huang,et al.  A GA-based feature selection and parameters optimizationfor support vector machines , 2006, Expert Syst. Appl..

[29]  Pascal Vincent,et al.  Kernel Matching Pursuit , 2002, Machine Learning.

[30]  Berthold Schweizer,et al.  Probabilistic Metric Spaces , 2011 .

[31]  Angus Macintyre,et al.  Trends in Logic , 2001 .

[32]  Marti A. Hearst.  Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..

[33]  Tomaso A. Poggio,et al.  Image Representations and Feature Selection for Multimedia Database Search , 2003, IEEE Trans. Knowl. Data Eng..

[34]  Ron Kohavi,et al.  Irrelevant Features and the Subset Selection Problem , 1994, ICML.

[35]  Qinghua Hu,et al.  Neighborhood rough set based heterogeneous feature subset selection , 2008, Inf. Sci..

[36]  Rajen B. Bhatt,et al.  On fuzzy-rough sets approach to feature selection , 2005, Pattern Recognit. Lett..

[37]  Wei-Zhi Wu,et al.  Attribute reduction based on evidence theory in incomplete decision systems , 2008, Inf. Sci..

[38]  Sayan Mukherjee,et al.  Feature Selection for SVMs , 2000, NIPS.

[39]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[40]  D. Dubois,et al.  ROUGH FUZZY SETS AND FUZZY ROUGH SETS , 1990 .

[41]  Siegfried Gottwald,et al.  Fuzzy Sets and Fuzzy Logic , 1993 .

[42]  Bernhard Moser,et al.  On Representing and Generating Kernels by Fuzzy Equivalence Relations , 2006, J. Mach. Learn. Res..

[43]  Wen-Xiu Zhang,et al.  An axiomatic characterization of a fuzzy generalization of rough sets , 2004, Inf. Sci..

[44]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[45]  Kezhi Mao,et al.  Feature subset selection for support vector machines through discriminative function pruning analysis , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[46]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[47]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[48]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[49]  Andrzej Skowron,et al.  The Discernibility Matrices and Functions in Information Systems , 1992, Intelligent Decision Support.

[50]  Nehad N. Morsi,et al.  Axiomatics for fuzzy rough sets , 1998, Fuzzy Sets Syst..

[51]  B. C. Brookes,et al.  Information Sciences , 2020, Cognitive Skills You Need for the 21st Century.

[52]  Didier Dubois,et al.  A review of fuzzy set aggregation connectives , 1985, Inf. Sci..

[53]  Qiang Shen,et al.  Semantics-preserving dimensionality reduction: rough and fuzzy-rough-based approaches , 2004, IEEE Transactions on Knowledge and Data Engineering.

[54]  Santosh S. Vempala,et al.  Kernels as features: On kernels, margins, and low-dimensional mappings , 2006, Machine Learning.