An Improved Second Order Training Algorithm for Improving the Accuracy of Fuzzy Decision Trees

Fuzzy decision tree FDT is a powerful top-down, hierarchical search methodology to extract human interpretable classification rules. The performance of FDT depends on initial fuzzy partitions and other parameters like alpha-cut and leaf selection threshold. These parameters are decided either heuristically or by trial-and-error. For given set of parameters, FDT is constructed using any standard induction algorithms like Fuzzy ID3. Due to the greedy nature of induction process, there is a chance of FDT resulting in poor classification accuracy. To further improve the accuracy of FDT, in this paper, the authors propose the strategy called Improved Second Order-Neuro-Fuzzy Decision Tree ISO-N-FDT. ISO-N-FDT tunes parameters of FDT from leaf node to roof node starting from left side of tree to its right and attains better improvement in accuracy with less number of iterations exhibiting fast convergence and powerful search ability.

[1]  Hyunjung Shin,et al.  wFDT - Weighted Fuzzy Decision Trees for Prognosis of Breast Cancer Survivability , 2008, AusDM.

[2]  Witold Pedrycz,et al.  Fuzzy rule based decision trees , 2015, Pattern Recognit..

[3]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[4]  Les M. Sztandera.  Fuzzy Neural Trees , 1996, Inf. Sci..

[5]  Shyi-Ming Chen,et al.  Fuzzy decision making and fuzzy group decision making based on likelihood-based comparison relations of hesitant fuzzy linguistic term sets , 2015, J. Intell. Fuzzy Syst..

[6]  Richard Weber,et al.  Fuzzy-ID3: A class of methods for automatic knowledge acquisition , 1992 .

[7]  M Gopal,et al.  On the structure and initial parameter identification of gaussian rbf networks , 2004, Int. J. Neural Syst..

[8]  B. Chandra,et al.  Fuzzy SLIQ Decision Tree Algorithm , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  Xizhao Wang,et al.  A comparative study on heuristic algorithms for generating fuzzy decision trees , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[10]  Hannu Koivisto,et al.  Fuzzy classifier identification using decision tree and multiobjective evolutionary algorithms , 2008, Int. J. Approx. Reason..

[11]  M. Shaw,et al.  Induction of fuzzy decision trees , 1995 .

[12]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[13]  Qiang Shen,et al.  Fuzzy-rough data reduction with ant colony optimization , 2005, Fuzzy Sets Syst..

[14]  Huayou Chen,et al.  Intuitionistic Fuzzy Interaction Bonferroni Means and Its Application to Multiple Attribute Decision Making , 2015, IEEE Transactions on Cybernetics.

[15]  Hao Yu,et al.  Improved Computation for Levenberg–Marquardt Training , 2010, IEEE Transactions on Neural Networks.

[16]  S. Wan,et al.  Trapezoidal intuitionistic fuzzy prioritized aggregation operators and application to multi-attribute decision making , 2015 .

[17]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[18]  Rajen B. Bhatt,et al.  FRCT: fuzzy-rough classification trees , 2007, Pattern Analysis and Applications.

[19]  Pei-Chann Chang,et al.  A CBR-based fuzzy decision tree approach for database classification , 2010, Expert Syst. Appl..

[20]  D. S. Yeung,et al.  Learning weighted fuzzy rules from examples with mixed attributes by fuzzy decision trees , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[21]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[22]  Hao Yu,et al.  Neural Network Learning Without Backpropagation , 2010, IEEE Transactions on Neural Networks.

[23]  Witold Pedrycz,et al.  C-fuzzy decision trees , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[24]  Rakesh Agrawal,et al.  SPRINT: A Scalable Parallel Classifier for Data Mining , 1996, VLDB.

[25]  Rajen B. Bhatt,et al.  Approximating fuzzy membership functions from clustered raw data , 2012, 2012 Annual IEEE India Conference (INDICON).

[26]  Xizhao Wang,et al.  On the optimization of fuzzy decision trees , 2000, Fuzzy Sets Syst..

[27]  J.W.T. Lee,et al.  Refinement of generated fuzzy production rules by using a fuzzy neural network , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Tao Wang,et al.  A survey of fuzzy decision tree classifier , 2009 .

[29]  Hao Yu,et al.  Fast and Efficient Second-Order Method for Training Radial Basis Function Networks , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[31]  Cezary Z. Janikow,et al.  Fuzzy decision trees: issues and methods , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[32]  L. Jain,et al.  Generation of fuzzy decision trees by fuzzy ID3 with adjusting mechanism of AND/OR operators , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[33]  Ilango Paramasivam,et al.  A Study on the Approximation of Clustered Data to Parameterized Family of Fuzzy Membership Functions for the Induction of Fuzzy Decision Trees , 2015 .

[34]  I. Hatono,et al.  Fuzzy decision trees by fuzzy ID3 algorithm and its application to diagnosis systems , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[35]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[36]  P.E. Maher,et al.  Uncertain reasoning in an ID3 machine learning framework , 1993, [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems.

[37]  Shyi-Ming Chen,et al.  Fuzzy decision making systems based on interval type-2 fuzzy sets , 2013, Inf. Sci..

[38]  Serge Guillaume,et al.  Designing fuzzy inference systems from data: An interpretability-oriented review , 2001, IEEE Trans. Fuzzy Syst..

[39]  Xizhao Wang,et al.  Improving learning accuracy of fuzzy decision trees by hybrid neural networks , 2000, IEEE Trans. Fuzzy Syst..

[40]  Jorma Rissanen,et al.  SLIQ: A Fast Scalable Classifier for Data Mining , 1996, EDBT.

[41]  Ishwar K. Sethi,et al.  Entropy nets: from decision trees to neural networks , 1990, Proc. IEEE.

[42]  Jane Yung-jen Hsu,et al.  Fuzzy classification trees for data analysis , 2002, Fuzzy Sets Syst..

[43]  Robert Ivor John,et al.  Interval-valued fuzzy decision trees with optimal neighbourhood perimeter , 2014, Appl. Soft Comput..

[44]  Rajen B. Bhatt,et al.  Neuro-fuzzy Decision Trees , 2006, Int. J. Neural Syst..

[45]  Bingchiang Jeng,et al.  FILM: a fuzzy inductive learning method for automated knowledge acquisition , 1997, Decis. Support Syst..

[46]  Shu-Ping Wan,et al.  Power Average of Trapezoidal Intuitionistic Fuzzy Numbers Using Strict t-Norms and t-Conorms , 2016, IEEE Transactions on Fuzzy Systems.

[47]  Jun-Hai Zhai,et al.  Fuzzy decision tree based on fuzzy-rough technique , 2011, Soft Comput..

[48]  Shyi-Ming Chen,et al.  Fuzzy decision making based on likelihood-based comparison relations of hesitant fuzzy linguistic term sets and hesitant fuzzy linguistic operators , 2015, Inf. Sci..

[49]  Huayou Chen,et al.  Hesitant Fuzzy Power Bonferroni Means and Their Application to Multiple Attribute Decision Making , 2015, IEEE Transactions on Fuzzy Systems.

[50]  F. Herrera,et al.  IIVFDT: ignorance functions based interval-valued fuzzy decision tree with genetic tuning , 2012 .

[51]  Xizhao Wang,et al.  On the handling of fuzziness for continuous-valued attributes in decision tree generation , 1998, Fuzzy Sets Syst..

[52]  Lukasz Bartczuk,et al.  Type-2 Fuzzy Decision Trees , 2006, ICAISC.

[53]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.