A comparative study on heuristic algorithms for generating fuzzy decision trees

Fuzzy decision tree induction is an important way of learning from examples with fuzzy representation. Since the construction of optimal fuzzy decision tree is NP-hard, the research on heuristic algorithms is necessary. In this paper, three heuristic algorithms for generating fuzzy decision trees are analyzed and compared. One of them is proposed by the authors. The comparisons are twofold. One is the analytic comparison based on expanded attribute selection and reasoning mechanism; the other is the experimental comparison based on the size of generated trees and learning accuracy. The purpose of this study is to explore comparative strengths and weaknesses of the three heuristics and to show some useful guidelines on how to choose an appropriate heuristic for a particular problem.

[1]  Daniel S. Yeung,et al.  A multilevel weighted fuzzy reasoning algorithm for expert systems , 1998, IEEE Trans. Syst. Man Cybern. Part A.

[2]  Maciej Wygralak.  Questions of cardinality of finite fuzzy sets , 1999, Fuzzy Sets Syst..

[3]  Ronald L. Rivest,et al.  Constructing Optimal Binary Decision Trees is NP-Complete , 1976, Inf. Process. Lett..

[4]  David A. Landgrebe,et al.  A survey of decision tree classifier methodology , 1991, IEEE Trans. Syst. Man Cybern..

[5]  Sankar K. Pal.  Fuzzy set theoretic measures for automatic feature evaluation: II , 1992, Inf. Sci..

[6]  Settimo Termini,et al.  A Definition of a Nonprobabilistic Entropy in the Setting of Fuzzy Sets Theory , 1972, Inf. Control..

[7]  G. Klir,et al.  MEASURES OF UNCERTAINTY AND INFORMATION BASED ON POSSIBILITY DISTRIBUTIONS , 1982 .

[8]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[9]  I. Hatono,et al.  Fuzzy decision trees by fuzzy ID3 algorithm and its application to diagnosis systems , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[10]  Raymond J. Mooney,et al.  Symbolic and Neural Learning Algorithms: An Experimental Comparison , 1991, Machine Learning.

[11]  M. Shaw,et al.  Induction of fuzzy decision trees , 1995 .

[12]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[13]  IchihashiH.,et al.  Neuro-fuzzy ID3 , 1996 .

[14]  George J. Klir,et al.  On the uniqueness of possibilistic measure of uncertainty and information , 1987 .

[15]  Hidetomo Ichihashi,et al.  Neuro-fuzzy ID3: a method of inducing fuzzy decision trees with linear programming for maximizing entropy and an algebraic method for incremental learning , 1996, Fuzzy Sets Syst..

[16]  Bingchiang Jeng,et al.  FILM: a fuzzy inductive learning method for automated knowledge acquisition , 1997, Decis. Support Syst..

[17]  J. Ross Quinlan,et al.  Decision trees and decision-making , 1990, IEEE Trans. Syst. Man Cybern..

[18]  Daniel S. Yeung,et al.  Weighted fuzzy production rules , 1997, Fuzzy Sets Syst..

[19]  D. S. Yeung,et al.  Learning weighted fuzzy rules from examples with mixed attributes by fuzzy decision trees , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[20]  Xizhao Wang,et al.  On the optimization of fuzzy decision trees , 2000, Fuzzy Sets Syst..

[21]  David Haussler,et al.  Occam's Razor , 1987, Inf. Process. Lett..

[22]  Z. Pawlak.  Rough Sets: Theoretical Aspects of Reasoning about Data , 1991 .

[23]  L. Zadeh.  Fuzzy sets as a basis for a theory of possibility , 1999 .

[24]  Richard Weber,et al.  Fuzzy-ID3: A class of methods for automatic knowledge acquisition , 1992 .

[25]  Daniel S. Yeung,et al.  A comparative study on similarity-based fuzzy reasoning methods , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[26]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[27]  Hisao Ishibuchi,et al.  A simple but powerful heuristic method for generating fuzzy rules from numerical data , 1997, Fuzzy Sets Syst..

[28]  Xizhao Wang,et al.  On the handling of fuzziness for continuous-valued attributes in decision tree generation , 1998, Fuzzy Sets Syst..