Identification of interpretable and accurate fuzzy classifiers and function estimators with hybrid methods

This paper studies the identification of fuzzy classifiers and function estimators focusing on improving their interpretability while maintaining their accuracy. Advances of various methods, such as, input variable selection, appropriate initialization algorithms, evolutionary algorithms and simplification techniques are hybridized to form a framework capable of identifying interpretable and accurate fuzzy models (FMs). FMs are initialized by two algorithms. Modified Gath-Geva (MGG) is used for function estimation and C4.5 for classification problems. The initialized FMs go through a three-step GA-based optimization, in which the adequate structure and parameters of FMs are searched. The proposed fitness function makes the favoring of simple FMs possible. Furthermore, the rule base is made more comprehensible by reducing the number of conditions in the rules. The validity of FMs is verified through studying several well-known benchmark problems. The results indicate, that by means of the proposed framework, interpretable, yet accurate FMs are obtained.

[1]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[2]  Francisco Herrera,et al.  Author's reply [to comments on 'A proposal to improve the accuracy of linguistic modelling'] , 2003, IEEE Trans. Fuzzy Syst..

[3]  Francisco Herrera,et al.  A genetic learning process for the scaling factors, granularity and contexts of the fuzzy rule-based system data base , 2001, Inf. Sci..

[4]  Joos Vandewalle,et al.  Predictive control using fuzzy models — Comparative study , 1999, 1999 European Control Conference (ECC).

[5]  Salvatore Ruggieri,et al.  Efficient C4.5 , 2002, IEEE Trans. Knowl. Data Eng..

[6]  János Abonyi,et al.  Learning Fuzzy Classification Rules from Data , 2001 .

[7]  María José del Jesús,et al.  Genetic tuning of fuzzy rule deep structures preserving interpretability and its interaction with fuzzy rule set reduction , 2005, IEEE Transactions on Fuzzy Systems.

[8]  David G. Stork,et al.  Pattern Classification , 1973 .

[9]  Hisao Ishibuchi,et al.  Three-objective genetics-based machine learning for linguistic rule extraction , 2001, Inf. Sci..

[10]  Gerardo Beni,et al.  A Validity Measure for Fuzzy Clustering , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[12]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[13]  Magne Setnes,et al.  GA-fuzzy modeling and classification: complexity and performance , 2000, IEEE Trans. Fuzzy Syst..

[14]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[15]  Sandip Sen,et al.  Using real-valued genetic algorithms to evolve rule sets for classification , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[16]  C. Tuzcu,et al.  An integrated multi-objective optimisation and fuzzy multi-attributive group decision-making technique for subdivision arrangement of Ro-Ro vessels , 2006, Appl. Soft Comput..

[17]  Uzay Kaymak,et al.  Similarity measures in fuzzy rule base simplification , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[18]  Heikki Mannila,et al.  Principles of Data Mining , 2001, Undergraduate Topics in Computer Science.

[19]  Magne Setnes,et al.  Compact and transparent fuzzy models and classifiers through iterative complexity reduction , 2001, IEEE Trans. Fuzzy Syst..

[20]  John Yen,et al.  Simplifying fuzzy rule-based models using orthogonal transformation methods , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[21]  E. Mizutani,et al.  Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence [Book Review] , 1997, IEEE Transactions on Automatic Control.

[22]  X. Shi,et al.  Invariant operators, small samples, and the bias-variance dilemma , 2004, CVPR 2004.

[23]  Ferenc Szeifert,et al.  Supervised fuzzy clustering for the identification of fuzzy classifiers , 2003, Pattern Recognit. Lett..

[24]  Jan Komorowski,et al.  Principles of Data Mining and Knowledge Discovery , 2001, Lecture Notes in Computer Science.

[25]  Ferenc Szeifert,et al.  Compact TS-fuzzy models through clustering and OLS plus FIS model reduction , 2001, 10th IEEE International Conference on Fuzzy Systems. (Cat. No.01CH37297).

[26]  Ujjwal Maulik,et al.  Performance Evaluation of Some Clustering Algorithms and Validity Indices , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Ferenc Szeifert,et al.  Data-driven generation of compact, accurate, and linguistically sound fuzzy classifiers based on a decision-tree initialization , 2003, Int. J. Approx. Reason..

[28]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[29]  Simon Parsons,et al.  Principles of Data Mining by David J. Hand, Heikki Mannila and Padhraic Smyth, MIT Press, 546 pp., £34.50, ISBN 0-262-08290-X , 2004, The Knowledge Engineering Review.

[30]  Qiangfu Zhao,et al.  Generation of comprehensible decision trees through evolution of training data , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[31]  Robert Babuska,et al.  Fuzzy Modeling for Control , 1998 .

[32]  Christian Setzkorn,et al.  On the use of multi-objective evolutionary algorithms for the induction of fuzzy classification rule systems. , 2005, Bio Systems.

[33]  George W. Irwin,et al.  Nonlinear control structures based on embedded neural system models , 1997, IEEE Trans. Neural Networks.

[34]  V. J. Rayward-Smith,et al.  Fuzzy Cluster Analysis: Methods for Classification, Data Analysis and Image Recognition , 1999 .

[35]  Vincent Wertz,et al.  Takagi-Sugeno fuzzy modeling incorporating input variables selection , 2002, IEEE Trans. Fuzzy Syst..

[36]  Francisco Herrera,et al.  Solving Electrical Distribution Problems Using Hybrid Evolutionary Data Analysis Techniques , 2004, Applied Intelligence.

[37]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[39]  Detlef D. Nauck,et al.  Fuzzy methods for automated intelligent data analysis , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[40]  Robert John,et al.  Developments in soft computing , 2001 .

[41]  Niels Kjølstad Poulsen,et al.  Neural Networks for Modelling and Control of Dynamic Systems: A Practitioner’s Handbook , 2000 .

[42]  Ferenc Szeifert,et al.  Modified Gath-Geva fuzzy clustering for identification of Takagi-Sugeno fuzzy models , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[43]  Petra Perner,et al.  Empirical Evaluation of Feature Subset Selection Based on a Real-World Data Set , 2000, PKDD.