Fuzzy Rough Decision Trees

How to evaluate features and select nodes is one of the key issues in constructing decision trees. In this work fuzzy rough set theory is employed to design an index for evaluating the quality of fuzzy features or numerical attributes. A fuzzy rough decision tree algorithm, which can be used to address classification problems described with symbolic, real-valued or fuzzy features, is developed. As node selection, split generation and stopping criterion are three main factors in constructing a decision tree, we design different techniques to determine splits with different kinds of features. The proposed algorithm can directly generate a classification tree without discretization or fuzzification of continuous attributes. Some numerical experiments are conducted and the comparative results show that the proposed algorithm is effective compared with some popular algorithms.

[1]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[2]  Xizhao Wang,et al.  On the optimization of fuzzy decision trees , 2000, Fuzzy Sets Syst..

[3]  Xizhao Wang,et al.  Attributes Reduction Using Fuzzy Rough Sets , 2008, IEEE Transactions on Fuzzy Systems.

[4]  Yiyu Yao,et al.  Attribute reduction in decision-theoretic rough set models , 2008, Inf. Sci..

[5]  David A. Landgrebe,et al.  A survey of decision tree classifier methodology , 1991, IEEE Trans. Syst. Man Cybern..

[6]  Xindong Wu,et al.  Error Detection and Impact-Sensitive Instance Ranking in Noisy Datasets , 2004, AAAI.

[7]  Alberto Suárez,et al.  Globally Optimal Fuzzy Decision Trees for Classification and Regression , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Hung Son Nguyen,et al.  Approximate Boolean Reasoning: Foundations and Applications in Data Mining , 2006, Trans. Rough Sets.

[9]  Xizhao Wang,et al.  Induction of multiple fuzzy decision trees based on rough set technique , 2008, Inf. Sci..

[10]  Qinghua Hu,et al.  Robust fuzzy rough classifiers , 2011, Fuzzy Sets Syst..

[11]  Richard Weber,et al.  Fuzzy-ID3: A class of methods for automatic knowledge acquisition , 1992 .

[12]  Chris Cornelis,et al.  Attribute selection with fuzzy decision reducts , 2010, Inf. Sci..

[13]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[14]  Qiang Shen,et al.  Fuzzy-Rough Feature Significance for Fuzzy Decision Trees , 2005 .

[15]  Wei-Zhi Wu,et al.  Generalized fuzzy rough sets , 2003, Inf. Sci..

[16]  Belur V. Dasarathy,et al.  Nearest neighbor (NN) norms: NN pattern classification techniques , 1991 .

[17]  Hui Xiong,et al.  Enhancing data analysis with noise removal , 2006, IEEE Transactions on Knowledge and Data Engineering.

[18]  D. S. Yeung,et al.  Learning weighted fuzzy rules from examples with mixed attributes by fuzzy decision trees , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[19]  Qiang Shen,et al.  Centre for Intelligent Systems and Their Applications Fuzzy Rough Attribute Reduction with Application to Web Categorization Fuzzy Rough Attribute Reduction with Application to Web Categorization Fuzzy Sets and Systems ( ) – Fuzzy–rough Attribute Reduction with Application to Web Categorization , 2022 .

[20]  C. Cornelis,et al.  Vaguely Quantified Rough Sets , 2009, RSFDGrC.

[21]  Xizhao Wang,et al.  On the generalization of fuzzy rough sets , 2005, IEEE Transactions on Fuzzy Systems.

[22]  Piero P. Bonissone,et al.  A fuzzy random forest , 2010, Int. J. Approx. Reason..

[23]  Xindong Wu,et al.  Class Noise Handling for Effective Cost-Sensitive Learning by Cost-Guided Iterative Classification Filtering , 2006, IEEE Transactions on Knowledge and Data Engineering.

[24]  Qinghua Hu,et al.  Soft Minimum-Enclosing-Ball Based Robust Fuzzy Rough Sets , 2012, Fundam. Informaticae.

[25]  Wen-Xiu Zhang,et al.  An axiomatic characterization of a fuzzy generalization of rough sets , 2004, Inf. Sci..

[26]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[27]  Wei-Zhi Wu,et al.  Constructive and axiomatic approaches of fuzzy approximation operators , 2004, Inf. Sci..

[28]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[29]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[30]  Louis Wehenkel,et al.  A complete fuzzy decision tree technique , 2003, Fuzzy Sets Syst..

[31]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[32]  Qinghua Hu,et al.  Soft fuzzy rough sets for robust feature evaluation and selection , 2010, Inf. Sci..

[33]  Rajen B. Bhatt,et al.  FRCT: fuzzy-rough classification trees , 2007, Pattern Analysis and Applications.

[34]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[35]  Anna Maria Radzikowska,et al.  A comparative study of fuzzy rough sets , 2002, Fuzzy Sets Syst..

[36]  Degang Chen,et al.  The Model of Fuzzy Variable Precision Rough Sets , 2009, IEEE Transactions on Fuzzy Systems.

[37]  Zied Elouedi,et al.  Classification systems based on rough sets under the belief function framework , 2011, Int. J. Approx. Reason..

[38]  Nehad N. Morsi,et al.  Axiomatics for fuzzy rough sets , 1998, Fuzzy Sets Syst..

[39]  Xindong Wu,et al.  Mining With Noise Knowledge: Error-Aware Data Mining , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[40]  Krzysztof J. Cios,et al.  Continuous ID3 algorithm with fuzzy entropy measures , 1992, [1992 Proceedings] IEEE International Conference on Fuzzy Systems.

[41]  Qinghua Hu,et al.  On Robust Fuzzy Rough Set Models , 2012, IEEE Transactions on Fuzzy Systems.

[42]  Xizhao Wang,et al.  A comparative study on heuristic algorithms for generating fuzzy decision trees , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[43]  M. Shaw,et al.  Induction of fuzzy decision trees , 1995 .

[44]  Qiang Shen,et al.  Semantics-preserving dimensionality reduction: rough and fuzzy-rough-based approaches , 2004, IEEE Transactions on Knowledge and Data Engineering.

[45]  Alicja Mieszkowicz-Rolka,et al.  Variable Precision Fuzzy Rough Sets , 2004, Trans. Rough Sets.

[46]  Haoyang Wu,et al.  An Interval Type-2 Fuzzy Rough Set Model for Attribute Reduction , 2009, IEEE Transactions on Fuzzy Systems.

[47]  D. Dubois,et al.  ROUGH FUZZY SETS AND FUZZY ROUGH SETS , 1990 .

[48]  Eyke Hüllermeier,et al.  Why Fuzzy Decision Trees are Good Rankers , 2009, IEEE Transactions on Fuzzy Systems.