Induction of fuzzy decision trees and its refinement using gradient projected-neuro-fuzzy decision tree

Fuzzy decision tree (FDT) induction is a powerful methodology to extract human interpretable classification rules. Due to the greedy nature of FDT, there is a chance of FDT resulting in poor classification accuracy. To improve the accuracy of FDT, Bhatt and Gopal (2006) have proposed a back propagation strategy, where the interpretability of derived fuzzy rules is affected, as the certainty factor of the rules does not lie within the theoretical bounds of 0 and 1. To retain the human interpretability of fuzzy rules, and to make rules consistent with fuzzy set theory, we restrict the values of certainty factor to lie within theoretical bounds using the concept of gradient projection over neuro fuzzy decision tree and the model is named as Gradient Projected-Neuro-Fuzzy Decision Tree (GP-N-FDT). Here, the parameters of FDT developed using Fuzzy ID3 algorithm are fine tuned using GP-N-FDT strategy to improve the classification accuracy.

[1]  Xizhao Wang,et al.  A comparative study on heuristic algorithms for generating fuzzy decision trees , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[2]  L. Jain,et al.  Generation of fuzzy decision trees by fuzzy ID3 with adjusting mechanism of AND/OR operators , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[3]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[4]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[5]  Gerardo Beni,et al.  A Validity Measure for Fuzzy Clustering , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Richard Weber,et al.  Fuzzy-ID3: A class of methods for automatic knowledge acquisition , 1992 .

[7]  M Gopal,et al.  On the structure and initial parameter identification of gaussian rbf networks , 2004, Int. J. Neural Syst..

[8]  Paul H. Calamai,et al.  Projected gradient methods for linearly constrained problems , 1987, Math. Program..

[9]  M. Shaw,et al.  Induction of fuzzy decision trees , 1995 .

[10]  Alberto Suárez,et al.  Globally Optimal Fuzzy Decision Trees for Classification and Regression , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Serge Guillaume,et al.  Designing fuzzy inference systems from data: An interpretability-oriented review , 2001, IEEE Trans. Fuzzy Syst..

[12]  Xizhao Wang,et al.  Improving learning accuracy of fuzzy decision trees by hybrid neural networks , 2000, IEEE Trans. Fuzzy Syst..

[13]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[14]  Krzysztof J. Cios,et al.  Continuous ID3 algorithm with fuzzy entropy measures , 1992, [1992 Proceedings] IEEE International Conference on Fuzzy Systems.

[15]  Wray L. Buntine Decision tree induction systems: A Bayesian analysis , 1987, Int. J. Approx. Reason..

[16]  Allen A. Goldstein,et al.  Constructive Real Analysis , 1967 .

[17]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[18]  Les M. Sztandera Fuzzy Neural Trees , 1996, Inf. Sci..

[19]  Ujjwal Maulik,et al.  Validity index for crisp and fuzzy clusters , 2004, Pattern Recognit..

[20]  E. Trauwaert On the meaning of Dunn's partition coefficient for fuzzy clusters , 1988 .

[21]  K. alik Cluster validity index for estimation of fuzzy clusters of different sizes and densities , 2010 .

[22]  D. S. Yeung,et al.  Learning weighted fuzzy rules from examples with mixed attributes by fuzzy decision trees , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[23]  I. Hatono,et al.  Fuzzy decision trees by fuzzy ID3 algorithm and its application to diagnosis systems , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[24]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[25]  Jonathan Lawry,et al.  Decision tree learning with fuzzy labels , 2005, Inf. Sci..

[26]  J.W.T. Lee,et al.  Refinement of generated fuzzy production rules by using a fuzzy neural network , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  Y. Fukuyama,et al.  A new method of choosing the number of clusters for the fuzzy c-mean method , 1989 .

[28]  E. Polak,et al.  Computational methods in optimization : a unified approach , 1972 .

[29]  Rajen B. Bhatt,et al.  Approximating fuzzy membership functions from clustered raw data , 2012, 2012 Annual IEEE India Conference (INDICON).

[30]  Xizhao Wang,et al.  On the optimization of fuzzy decision trees , 2000, Fuzzy Sets Syst..

[31]  Ishwar K. Sethi,et al.  Entropy nets: from decision trees to neural networks , 1990, Proc. IEEE.

[32]  Jane Yung-jen Hsu,et al.  Fuzzy classification trees for data analysis , 2002, Fuzzy Sets Syst..

[33]  Rajen B. Bhatt,et al.  Neuro-fuzzy Decision Trees , 2006, Int. J. Neural Syst..

[34]  Bingchiang Jeng,et al.  FILM: a fuzzy inductive learning method for automated knowledge acquisition , 1997, Decis. Support Syst..

[35]  B. Kosko Fuzziness vs. probability , 1990 .

[36]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[37]  A. Goldstein Convex programming in Hilbert space , 1964 .

[38]  Cezary Z. Janikow,et al.  Fuzzy decision trees: issues and methods , 1998, IEEE Trans. Syst. Man Cybern. Part B.