A multi-objective evolutionary method for learning granularities based on fuzzy discretization to improve the accuracy-complexity trade-off of fuzzy rule-based classification systems: D-MOFARC algorithm

HighlightsThis contribution presents a fuzzy discretization procedure.The procedure is used for learning granularities and fuzzy partitions of fuzzy rule based systems.This procedure is integrated within a multi-objective evolutionary algorithm (MOEA).The MOEA concurrently performs a tuning and a rule selection process.The aim of the overall method is to improve the complexity-accuracy trade-off of fuzzy models. Multi-objective evolutionary algorithms represent an effective tool to improve the accuracy-interpretability trade-off of fuzzy rule-based classification systems. To this aim, a tuning process and a rule selection process can be combined to obtain a set of solutions with different trade-offs between the accuracy and the compactness of models. Nevertheless, an initial model needs to be defined, in particular the parameters that describe the partitions and the number of fuzzy sets of each variable (i.e. the granularities) must be determined. The simplest approach is to use a previously established single granularity and a uniform fuzzy partition for each variable. A better approach consists in automatically identifying from data the appropriate granularities and fuzzy partitions, since this usually leads to more accurate models.This contribution presents a fuzzy discretization approach, which is used to generate automatically promising granularities and their associated fuzzy partitions. This mechanism is integrated within a Multi-Objective Fuzzy Association Rule-Based Classification method, namely D-MOFARC, which concurrently performs a tuning and a rule selection process on an initial knowledge base. The aim is to obtain fuzzy rule-based classification systems with high classification performances, while preserving their complexity.

[1]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[2]  Francisco Herrera,et al.  Multiobjective genetic fuzzy rule selection of single granularity-based fuzzy classification rules and its interaction with the lateral tuning of membership functions , 2011, Soft Comput..

[3]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[4]  Francisco Herrera,et al.  A Review of the Application of Multiobjective Evolutionary Fuzzy Systems: Current Status and Further Directions , 2013, IEEE Transactions on Fuzzy Systems.

[5]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[6]  Francisco Alfredo Márquez,et al.  Parallel Distributed Two-Level Evolutionary Multiobjective Methodology for Granularity Learning and Membership Functions Tuning in Linguistic Fuzzy Systems , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[7]  Oscar Castillo,et al.  An Analysis on the Intrinsic Implementation of the Principle of Justifiable Granularity in Clustering Algorithms , 2013, Recent Advances on Hybrid Intelligent Systems.

[8]  David J. Sheskin,et al.  Handbook of Parametric and Nonparametric Statistical Procedures , 1997 .

[9]  Branko Kavsek,et al.  APRIORI-SD: ADAPTING ASSOCIATION RULE LEARNING TO SUBGROUP DISCOVERY , 2006, IDA.

[10]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[11]  Francisco Herrera,et al.  A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability , 2009, Soft Comput..

[12]  Larry J. Eshelman,et al.  The CHC Adaptive Search Algorithm: How to Have Safe Search When Engaging in Nontraditional Genetic Recombination , 1990, FOGA.

[13]  Francisco Herrera,et al.  A Multiobjective Evolutionary Approach to Concurrently Learn Rule and Data Bases of Linguistic Fuzzy-Rule-Based Systems , 2009, IEEE Transactions on Fuzzy Systems.

[14]  Francisco Herrera,et al.  Linguistic modeling with hierarchical systems of weighted linguistic rules , 2003, Int. J. Approx. Reason..

[15]  Lukasz A. Kurgan,et al.  CAIM discretization algorithm , 2004, IEEE Transactions on Knowledge and Data Engineering.

[16]  Hisao Ishibuchi,et al.  Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining , 2004, Fuzzy Sets Syst..

[17]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[18]  Francisco Herrera,et al.  Genetic Fuzzy Systems - Evolutionary Tuning and Learning of Fuzzy Knowledge Bases , 2002, Advances in Fuzzy Systems - Applications and Theory.

[19]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[20]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic and Evolutionary Computation) , 2006 .

[21]  Witold Pedrycz,et al.  The Principle of Justifiable Granularity and an Optimization of Information Granularity Allocation as Fundamentals of Granular Computing , 2011, J. Inf. Process. Syst..

[22]  Shichao Zhang,et al.  Association Rule Mining: Models and Algorithms , 2002 .

[23]  Beatrice Lazzerini,et al.  Multi-objective evolutionary learning of granularity, membership function parameters and rules of Mamdani fuzzy systems , 2009, Evol. Intell..

[24]  J. Casillas.  Interpretability issues in fuzzy modeling , 2003 .

[25]  Jesús Alcalá-Fdez,et al.  A Fuzzy Association Rule-Based Classification Model for High-Dimensional Problems With Genetic Rule Selection and Lateral Tuning , 2011, IEEE Transactions on Fuzzy Systems.

[26]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[27]  Hisao Ishibuchi,et al.  Classification and modeling with linguistic information granules - advanced approaches to linguistic data mining , 2004, Advanced information processing.

[28]  Witold Pedrycz,et al.  A genetic design of linguistic terms for fuzzy rule based classifiers , 2013, Int. J. Approx. Reason..

[29]  Francisco Herrera,et al.  Analysis and guidelines to obtain a good uniform fuzzy partition granularity for fuzzy rule-based systems using simulated annealing , 2000, Int. J. Approx. Reason..

[30]  Jason Catlett,et al.  On Changing Continuous Attributes into Ordered Discrete Attributes , 1991, EWSL.

[31]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[32]  W. Pedrycz,et al.  Construction of fuzzy models through clustering techniques , 1993 .

[33]  Francisco Herrera,et al.  Adaptation and application of multi-objective evolutionary algorithms for rule reduction and parameter tuning of fuzzy rule-based systems , 2008, Soft Comput..

[34]  Francisco Herrera,et al.  Generating the knowledge base of a fuzzy rule-based system by the genetic learning of the data base , 2001, IEEE Trans. Fuzzy Syst..

[35]  Hisao Ishibuchi,et al.  Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems , 1997, Fuzzy Sets Syst..

[36]  Witold Pedrycz,et al.  Design of interval type-2 fuzzy models through optimal granularity allocation , 2011, Appl. Soft Comput..

[37]  Francisco Herrera,et al.  Genetic fuzzy systems: taxonomy, current research trends and prospects , 2008, Evol. Intell..

[38]  Beatrice Lazzerini,et al.  Learning concurrently partition granularities and rule bases of Mamdani fuzzy systems in a multi-objective evolutionary framework , 2009, Int. J. Approx. Reason..

[39]  Michela Antonelli,et al.  An efficient multi-objective evolutionary fuzzy system for regression problems , 2013, Int. J. Approx. Reason..

[40]  Usama M. Fayyad,et al.  On the Handling of Continuous-Valued Attributes in Decision Tree Generation , 1992, Machine Learning.

[41]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[42]  D. Sheskin.  Handbook of parametric and nonparametric statistical procedures, 2nd ed. , 2000 .