Multiobjective Genetic Fuzzy Systems: Review and Future Research Directions

Evolutionary algorithms have been successfully used in many studies to design accurate and interpretable fuzzy systems under the name of genetic fuzzy systems. Recently evolutionary multiobjective algorithms have been used for interpretability-accuracy tradeoff analysis of fuzzy systems. We first review a wide range of related studies to multiobjective genetic fuzzy systems. Then we illustrate multiobjective design of fuzzy systems through computational experiments on some benchmark data sets. Finally we point out promising future research directions.

[1]  Tzung-Pei Hong,et al.  Trade-off Between Computation Time and Number of Rules for Fuzzy Mining from Quantitative Data , 2001, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[2]  Bernhard Sendhoff,et al.  On generating FC3 fuzzy rule systems from data using evolution strategies , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[3]  Peter J. Fleming,et al.  Multiobjective Genetic Programming: A Nonlinear System Identification Application , 1997 .

[4]  Hiroshi Motoda,et al.  Feature Selection for Knowledge Discovery and Data Mining , 1998, The Springer International Series in Engineering and Computer Science.

[5]  Hisao Ishibuchi,et al.  Optimization of Scalarizing Functions Through Evolutionary Multiobjective Optimization , 2007, EMO.

[6]  W. Peizhuang.  Pattern Recognition with Fuzzy Objective Function Algorithms (James C. Bezdek) , 1983 .

[7]  Matthew A. Kupinski,et al.  Multiobjective Genetic Optimization of Diagnostic Classifiers with Implications for Generating ROC Curves , 1999, IEEE Trans. Medical Imaging.

[8]  Beatriz de la Iglesia,et al.  Rule Induction for Classification Using Multi-objective Genetic Programming , 2007, EMO.

[9]  Heikki Mannila,et al.  Verkamo: Fast Discovery of Association Rules , 1996, KDD 1996.

[10]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[11]  Kalyanmoy Deb,et al.  Reference point based multi-objective optimization using evolutionary algorithms , 2006, GECCO.

[12]  Francisco Herrera,et al.  Stratification for scaling up evolutionary prototype selection , 2005, Pattern Recognit. Lett..

[13]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[14]  Ralf Mikut,et al.  Interpretability issues in data-based learning of fuzzy systems , 2005, Fuzzy Sets Syst..

[15]  Yoshiki Uchikawa,et al.  On fuzzy modeling using fuzzy neural networks with the back-propagation algorithm , 1992, IEEE Trans. Neural Networks.

[16]  Detlef Nauck,et al.  Foundations Of Neuro-Fuzzy Systems , 1997 .

[17]  Luis Magdalena,et al.  A Multiobjective Genetic Learning Process for joint Feature Selection and Granularity and Contexts Learning in Fuzzy Rule-Based Classification Systems , 2003 .

[18]  Hussein A. Abbass,et al.  Pareto neuro-evolution: constructing ensemble of neural networks using multi-objective optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[19]  Francisco Herrera,et al.  Linguistic modeling with hierarchical systems of weighted linguistic rules , 2003, Int. J. Approx. Reason..

[20]  Antonio González Muñoz,et al.  SLAVE: a genetic learning system based on an iterative approach , 1999, IEEE Trans. Fuzzy Syst..

[21]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[22]  Joshua D. Knowles,et al.  Evolutionary Multiobjective Clustering , 2004, PPSN.

[23]  Victor J. Rayward-Smith,et al.  The application and effectiveness of a multi-objective metaheuristic algorithm for partial classification , 2006, Eur. J. Oper. Res..

[24]  Antonio González Muñoz,et al.  Including a simplicity criterion in the selection of the best rule in a genetic fuzzy learning algorithm , 2001, Fuzzy Sets Syst..

[25]  Bernhard Sendhoff,et al.  Neural network regularization and ensembling using multi-objective evolutionary algorithms , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[26]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[27]  C. L. Karr,et al.  Fuzzy control of pH using genetic algorithms , 1993, IEEE Trans. Fuzzy Syst..

[28]  Beatriz de la Iglesia,et al.  Rule Induction Using Multi-Objective Metaheuristics: Encouraging Rule Diversity , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[29]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[30]  Joshua D. Knowles,et al.  Improvements to the scalability of multiobjective clustering , 2005, 2005 IEEE Congress on Evolutionary Computation.

[31]  DebK.,et al.  A fast and elitist multiobjective genetic algorithm , 2002 .

[32]  Luiz Eduardo Soares de Oliveira,et al.  Multi-objective Genetic Algorithms to Create Ensemble of Classifiers , 2005, EMO.

[33]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[34]  Bernhard Sendhoff,et al.  Evolutionary Multi-objective Optimization for Simultaneous Generation of Signal-Type and Symbol-Type Representations , 2005, EMO.

[35]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[36]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[37]  Hisao Ishibuchi,et al.  Classification and modeling with linguistic information granules - advanced approaches to linguistic data mining , 2004, Advanced information processing.

[38]  L X Wang,et al.  Fuzzy basis functions, universal approximation, and orthogonal least-squares learning , 1992, IEEE Trans. Neural Networks.

[39]  Mehmet Kaya,et al.  Multi-objective genetic algorithm based approaches for mining optimized fuzzy association rules , 2006, Soft Comput..

[40]  Ferenc Szeifert,et al.  Data-driven generation of compact, accurate, and linguistically sound fuzzy classifiers based on a decision-tree initialization , 2003, Int. J. Approx. Reason..

[41]  Thomas G. Dietterich.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[42]  María José del Jesús,et al.  MOGUL: A methodology to obtain genetic fuzzy rule-based systems under the iterative rule learning approach , 1999, Int. J. Intell. Syst..

[43]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[44]  J. Casillas.  Interpretability issues in fuzzy modeling , 2003 .

[45]  Oscar Cordón,et al.  Evolutionary Learning of Boolean Queries by Multiobjective Genetic Programming , 2002, PPSN.

[46]  Yaochu Jin,et al.  Multi-Objective Machine Learning , 2006, Studies in Computational Intelligence.

[47]  Heikki Mannila,et al.  Fast Discovery of Association Rules , 1996, Advances in Knowledge Discovery and Data Mining.

[48]  Kim-Fung Man,et al.  Agent-based evolutionary approach for interpretable rule-based knowledge extraction , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[49]  D.E. Goldberg,et al.  Classifier Systems and Genetic Algorithms , 1989, Artif. Intell..

[50]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[51]  Hisao Ishibuchi,et al.  Three-objective genetics-based machine learning for linguistic rule extraction , 2001, Inf. Sci..

[52]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[53]  M. Anastasio,et al.  Multiobjective genetic optimization of diagnostic classifiers with implications for generating receiver operating characteristic curves , 1999, IEEE Transactions on Medical Imaging.

[54]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[55]  Hisao Ishibuchi,et al.  Hybridization of fuzzy GBML approaches for pattern classification problems , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[56]  Satchidananda Dehuri,et al.  Multi-Objective Evolutionary Algorithms for Knowledge Discovery from Databases , 2008, Multi-Objective Evolutionary Algorithms for Knowledge Discovery from Databases.

[57]  Bart Kosko,et al.  Fuzzy Systems as Universal Approximators , 1994, IEEE Trans. Computers.

[58]  Hisao Ishibuchi,et al.  Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning , 2007, Int. J. Approx. Reason..

[59]  María José del Jesús,et al.  Genetic tuning of fuzzy rule deep structures preserving interpretability and its interaction with fuzzy rule set reduction , 2005, IEEE Transactions on Fuzzy Systems.

[60]  David S. Feldman,et al.  Fuzzy Network Synthesis with Genetic Algorithms , 1993, ICGA.

[61]  Philip R. Thrift,et al.  Fuzzy Logic Synthesis with Genetic Algorithms , 1991, ICGA.

[62]  Alexandre Parodi,et al.  A New Approach to Fuzzy Classifier Systems , 1993, ICGA.

[63]  阿部 重夫.  Pattern classification : neuro-fuzzy methods and their comparison , 2001 .

[64]  Victor J. Rayward-Smith,et al.  Developments on a Multi-objective Metaheuristic (MOMH) Algorithm for Finding Interesting Sets of Classification Rules , 2005, EMO.

[65]  John Yen,et al.  Improving the interpretability of TSK fuzzy models by combining global learning and local learning , 1998, IEEE Trans. Fuzzy Syst..

[66]  Magne Setnes,et al.  GA-fuzzy modeling and classification: complexity and performance , 2000, IEEE Trans. Fuzzy Syst..

[67]  Yi-Chung Hu,et al.  Finding fuzzy classification rules using data mining techniques , 2003, Pattern Recognit. Lett..

[68]  Lothar Thiele,et al.  Multiobjective genetic programming: reducing bloat using SPEA2 , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[69]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[70]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[71]  Serge Guillaume,et al.  Designing fuzzy inference systems from data: An interpretability-oriented review , 2001, IEEE Trans. Fuzzy Syst..

[72]  Hisao Ishibuchi,et al.  Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems , 1997, Fuzzy Sets Syst..

[73]  F. Herrera,et al.  Accuracy Improvements in Linguistic Fuzzy Modeling , 2003 .

[74]  M. Lozano,et al.  MOGUL: A methodology to obtain genetic fuzzy rule‐based systems under the iterative rule learning approach , 1999 .

[75]  Hisao Ishibuchi,et al.  Finding Simple Fuzzy Classification Systems with High Interpretability Through Multiobjective Rule Selection , 2006, KES.

[76]  Hussein A. Abbass,et al.  Speeding Up Backpropagation Using Multiobjective Evolutionary Algorithms , 2003, Neural Computation.

[77]  Héctor Pomares,et al.  Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation , 2003, IEEE Trans. Neural Networks.

[78]  Francisco Herrera,et al.  Genetic Fuzzy Systems: Status, Critical Considerations and Future Directions , 2005 .

[79]  Evan J. Hughes,et al.  Evolutionary many-objective optimisation: many once or one many? , 2005, 2005 IEEE Congress on Evolutionary Computation.

[80]  Rudolf Kruse,et al.  Obtaining interpretable fuzzy classification rules from medical data , 1999, Artif. Intell. Medicine.

[81]  Xavier Llorà,et al.  Bounding the Effect of Noise in Multiobjective Learning Classifier Systems , 2003, Evolutionary Computation.

[82]  Xin Yao,et al.  Evolutionary framework for the construction of diverse hybrid ensembles , 2005, ESANN.

[83]  Manuel Valenzuela-Rendón,et al.  The Fuzzy Classifier System: A Classifier System for Continuously Varying Variables , 1991, ICGA.

[84]  Joshua D. Knowles,et al.  Exploiting the Trade-off - The Benefits of Multiple Objectives in Data Clustering , 2005, EMO.

[85]  Luiz Eduardo Soares de Oliveira,et al.  Feature selection for ensembles:a hierarchical multi-objective genetic algorithm approach , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[86]  Andrzej Jaszkiewicz,et al.  On the computational efficiency of multiple objective metaheuristics. The knapsack problem case study , 2004, Eur. J. Oper. Res..

[87]  Luiz Eduardo Soares de Oliveira,et al.  A Methodology for Feature Selection Using Multiobjective Genetic Algorithms for Handwritten Digit String Recognition , 2003, Int. J. Pattern Recognit. Artif. Intell..

[88]  Francisco Herrera,et al.  Ten years of genetic fuzzy systems: current framework and new trends , 2004, Fuzzy Sets Syst..

[89]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[90]  Luiz Eduardo Soares de Oliveira,et al.  Feature selection using multi-objective genetic algorithms for handwritten digit recognition , 2002, Object recognition supported by user interaction for service robots.

[91]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[92]  Vladimir Cherkassky,et al.  Learning from Data: Concepts, Theory, and Methods , 1998 .

[93]  Kim-Fung Man,et al.  Multi-objective hierarchical genetic algorithm for interpretable fuzzy rule-based knowledge extraction , 2005, Fuzzy Sets Syst..

[94]  Magne Setnes,et al.  Rule-based modeling: precision and transparency , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[95]  Hisao Ishibuchi,et al.  Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining , 2004, Fuzzy Sets Syst..

[96]  C. Emmanouilidis,et al.  A multiobjective evolutionary setting for feature selection and a commonality-based crossover operator , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[97]  Yaochu Jin,et al.  Fuzzy modeling of high-dimensional systems: complexity reduction and interpretability improvement , 2000, IEEE Trans. Fuzzy Syst..

[98]  Xin Yao,et al.  DIVACE: Diverse and Accurate Ensemble Learning Algorithm , 2004, IDEAL.

[99]  Joshua D. Knowles,et al.  Multiobjective clustering around medoids , 2005, 2005 IEEE Congress on Evolutionary Computation.

[100]  Lourdes Araujo,et al.  Multiobjective Genetic Programming for Natural Language Parsing and Tagging , 2006, PPSN.

[101]  Bhabesh Nath,et al.  Multi-objective rule mining using genetic algorithms , 2004, Inf. Sci..

[102]  J. Mendel.  Fuzzy logic systems for engineering: a tutorial , 1995, Proc. IEEE.

[103]  V. J. Rayward-Smith,et al.  Data mining rules using multi-objective evolutionary algorithms , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[104]  Hisao Ishibuchi,et al.  Evolutionary Multiobjective Optimization for Generating an Ensemble of Fuzzy Rule-Based Classifiers , 2003, GECCO.

[105]  Magne Setnes,et al.  Compact and transparent fuzzy models and classifiers through iterative complexity reduction , 2001, IEEE Trans. Fuzzy Syst..

[106]  Roberto J. Bayardo,et al.  Mining the most interesting rules , 1999, KDD '99.

[107]  Dr. Alex A. Freitas.  Data Mining and Knowledge Discovery with Evolutionary Algorithms , 2002, Natural Computing Series.

[108]  Hisao Ishibuchi,et al.  Evolutionary Multiobjective Knowledge Extraction for High-Dimensional Pattern Classification Problems , 2004, PPSN.

[109]  Hisao Ishibuchi,et al.  Relation between Pareto-Optimal Fuzzy Rules and Pareto-Optimal Fuzzy Rule Sets , 2007, 2007 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making.

[110]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[111]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.