On constructing a fuzzy inference framework using crisp decision trees

This paper proposes a framework which consists of a novel fuzzy inference algorithm to generate fuzzy decision trees from induced crisp decision trees. Fuzzy theoretical techniques are used to fuzzify crisp decision trees in order to soften sharp decision boundaries at decision nodes inherent in this type of trees. A framework for the investigation of various types of membership functions and fuzzy inference techniques is proposed. Once the decision tree has been fuzzified, all branches throughout the tree will fire, resulting in a membership grade being generated at each branch. Five different fuzzy inference mechanisms are used to investigate the degree of interaction between membership grades on each path in the decision tree, which ultimately leads to a final crisp classification. A genetic algorithm is used to optimize and automatically determine the set of fuzzy regions for all branches and simultaneously the degree in which the inference parameters will be applied. Comparisons between crisp trees and the fuzzified trees suggest that the later fuzzy tree is significantly more robust and produces a more balanced classification. In addition, the results obtained from five real-world data sets show that there is a significant improvement in the accuracy of the fuzzy trees when compared with crisp trees.

[1]  Richard Weber,et al.  Fuzzy-ID3: A class of methods for automatic knowledge acquisition , 1992 .

[2]  Christophe Marsala,et al.  Application of Fuzzy Rule Induction to Data Mining , 1998, FQAS.

[3]  Shyi-Ming Chen,et al.  GENERATING FUZZY RULES FROM TRAINING DATA CONTAINING NOISE FOR HANDLING CLASSIFICATION PROBLEMS , 2002 .

[4]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[5]  A. Townsend Genetic Algorithms – a Tutorial , 2003 .

[6]  Paul Compton,et al.  Inductive knowledge acquisition: a case study , 1987 .

[7]  Bernadette Bouchon-Meunier,et al.  Learning from Imperfect Data , 1997 .

[8]  Dimiter Driankov,et al.  Fuzzy model identification - selected approaches , 1997 .

[9]  Ron Kohavi,et al.  The Case against Accuracy Estimation for Comparing Induction Algorithms , 1998, ICML.

[10]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[11]  Kazuo Tanaka,et al.  Fuzzy modeling by ID3 algorithm and its application to prediction of heater outlet temperature , 1992, [1992 Proceedings] IEEE International Conference on Fuzzy Systems.

[12]  Earl Cox,et al.  The fuzzy systems handbook , 1994 .

[13]  E. Chong,et al.  Fuzzy modeling by induction and pruning of decision trees , 1994, Proceedings of 1994 9th IEEE International Symposium on Intelligent Control.

[14]  J. Dombi A general class of fuzzy operators, the demorgan class of fuzzy operators and fuzziness measures induced by fuzzy operators , 1982 .

[15]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[16]  Keon-Myung Lee,et al.  Fuzzy Information Processing for Expert Systems , 1995, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[17]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[18]  Derek A. Linkens,et al.  Rule-base self-generation and simplification for data-driven fuzzy models , 2001, 10th IEEE International Conference on Fuzzy Systems. (Cat. No.01CH37297).

[19]  Andrea Bonarini,et al.  Evolutionary Learning of Fuzzy rules: competition and cooperation , 1996 .

[20]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[21]  J. Ross Quinlan,et al.  Improved Use of Continuous Attributes in C4.5 , 1996, J. Artif. Intell. Res..

[22]  Piero P. Bonissone,et al.  Editorial: Fuzzy information processing , 1984, Inf. Sci..

[23]  Shyi-Ming Chen,et al.  GENERATING FUZZY RULES FROM TRAINING DATA CONTAINING NOISE FOR HANDLING CLASSIFICATION PROBLEMS , 2002, Cybern. Syst..

[24]  B. Bouchon-Meunier,et al.  Fuzzy partitioning using mathematical morphology in a learning scheme , 1996, Proceedings of IEEE 5th International Fuzzy Systems.

[25]  M. Fajfer,et al.  Fuzzy partitioning with FID3.1 , 1999, 18th International Conference of the North American Fuzzy Information Processing Society - NAFIPS (Cat. No.99TH8397).

[26]  Francisco Herrera,et al.  Ten years of genetic fuzzy systems: current framework and new trends , 2004, Fuzzy Sets Syst..

[27]  Zbigniew Michalewicz,et al.  Genetic Algorithms Plus Data Structures Equals Evolution Programs , 1994 .

[28]  Geoffrey I. Webb,et al.  On the Application of ROC Analysis to Predict Classification Performance Under Varying Class Distributions , 2005, Machine Learning.

[29]  Ching-Chang Wong,et al.  Design of fuzzy classification system using genetic algorithms , 2000, Ninth IEEE International Conference on Fuzzy Systems. FUZZ- IEEE 2000 (Cat. No.00CH37063).

[30]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[31]  O. Mangasarian,et al.  Multisurface method of pattern separation for medical diagnosis applied to breast cytology. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[32]  Cezary Z. Janikow,et al.  Fuzzy decision trees: issues and methods , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[33]  J A Swets,et al.  Measuring the accuracy of diagnostic systems. , 1988, Science.

[34]  Louis Wehenkel,et al.  A complete fuzzy decision tree technique , 2003, Fuzzy Sets Syst..

[35]  Didier Dubois,et al.  A class of fuzzy measures based on triangular norms , 1982 .

[36]  Cezary Z. Janikow,et al.  Exemplar learning in fuzzy decision trees , 1996, Proceedings of IEEE 5th International Fuzzy Systems.

[37]  Derek A. Linkens,et al.  Rule-base self-generation and simplification for data-driven fuzzy models , 2004, Fuzzy Sets Syst..

[38]  George J. Klir,et al.  Fuzzy sets, uncertainty and information , 1988 .

[39]  Oscar Cordón,et al.  Identification of linguistic fuzzy models by means of genetic algorithms , 1997 .

[40]  Jørgen Hilden,et al.  What properties should an overall measure of test performance possess? , 2005, Clinical chemistry.

[41]  J. R. Quinlan Probabilistic decision trees , 1990 .

[42]  Charles L. Karr,et al.  Improved Fuzzy Process Control of Spacecraft Autonomous Rendezvous Using a Genetic Algorithm , 1990, Other Conferences.

[43]  Didier Dubois,et al.  Fuzzy information engineering: a guided tour of applications , 1997 .

[44]  Francisco Herrera,et al.  Genetic Fuzzy Systems - Evolutionary Tuning and Learning of Fuzzy Knowledge Bases , 2002, Advances in Fuzzy Systems - Applications and Theory.

[45]  Giovanna Castellano,et al.  A staged approach for generation and compression of fuzzy classification rules , 2000, Ninth IEEE International Conference on Fuzzy Systems. FUZZ- IEEE 2000 (Cat. No.00CH37063).

[46]  George Singer,et al.  A genetic algorithm based fuzzy logic controller for non-linear systems , 1997 .

[47]  M. Shaw,et al.  Induction of fuzzy decision trees , 1995 .

[48]  R. Yager On a general class of fuzzy connectives , 1980 .

[49]  Chris Carter,et al.  Assessing Credit Card Applications Using Machine Learning , 1987, IEEE Expert.

[50]  Sankar K. Pal,et al.  Genetic Algorithms for Pattern Recognition , 2017 .