Fuzzy Data Mining by Heuristic Rule Extraction and Multiobjective Genetic Rule Selection

In this paper, we demonstrate that multiobjective genetic rule selection can significantly improve the accuracy-complexity tradeoff curve of fuzzy rule-based classification systems generated by a heuristic rule extraction procedure for classification problems with many continuous attributes. First a prespecifled number of fuzzy rules are extracted in a heuristic manner based on a rule evaluation criterion. This step can be viewed as fuzzy data mining. Then multiobjective genetic rule selection is applied to the extracted rules to find a number of non-dominated rule sets with respect to accuracy maximization and complexity minimization. This step can be viewed as a postprocessing procedure in fuzzy data mining. Experimental results show that multiobjective genetic rule selection finds a number of smaller rule sets with higher classification accuracy than heuristically extracted rule sets. That is, the accuracy-complexity tradeoff curve of heuristically extracted rule sets in fuzzy data mining is improved by multiobjective genetic rule selection. This observation suggests that multiobjective genetic rule selection plays an important role in fuzzy data mining as a postprocessing procedure.

[1]  Hisao Ishibuchi,et al.  Classification and modeling with linguistic information granules - advanced approaches to linguistic data mining , 2004, Advanced information processing.

[2]  Hussein A. Abbass,et al.  Speeding Up Backpropagation Using Multiobjective Evolutionary Algorithms , 2003, Neural Computation.

[3]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[4]  Heikki Mannila,et al.  Fast Discovery of Association Rules , 1996, Advances in Knowledge Discovery and Data Mining.

[5]  Gary B. Lamont,et al.  Applications Of Multi-Objective Evolutionary Algorithms , 2004 .

[6]  Kim-Fung Man,et al.  Multi-objective hierarchical genetic algorithm for interpretable fuzzy rule-based knowledge extraction , 2005, Fuzzy Sets Syst..

[7]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[8]  Kim-Fung Man,et al.  Agent-based evolutionary approach for interpretable rule-based knowledge extraction , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Hideo Tanaka,et al.  Construction of fuzzy classification systems with rectangular fuzzy rules using genetic algorithms , 1994, CVPR 1994.

[10]  Yaochu Jin,et al.  Multi-Objective Machine Learning , 2006, Studies in Computational Intelligence.

[11]  Hisao Ishibuchi,et al.  Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems , 1997, Fuzzy Sets Syst..

[12]  Héctor Pomares,et al.  Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation , 2003, IEEE Trans. Neural Networks.

[13]  Tapio Elomaa,et al.  General and Efficient Multisplitting of Numerical Attributes , 1999, Machine Learning.

[14]  Hisao Ishibuchi,et al.  Rule weight specification in fuzzy rule-based classification systems , 2005, IEEE Transactions on Fuzzy Systems.

[15]  Hisao Ishibuchi,et al.  Three-objective genetics-based machine learning for linguistic rule extraction , 2001, Inf. Sci..

[16]  Hussein A. Abbass,et al.  Pareto neuro-evolution: constructing ensemble of neural networks using multi-objective optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[17]  Xin Yao,et al.  Evolutionary framework for the construction of diverse hybrid ensembles , 2005, ESANN.

[18]  Xavier Llorà,et al.  Bounding the Effect of Noise in Multiobjective Learning Classifier Systems , 2003, Evolutionary Computation.

[19]  Hisao Ishibuchi,et al.  Evolutionary Multiobjective Optimization for Generating an Ensemble of Fuzzy Rule-Based Classifiers , 2003, GECCO.

[20]  María José del Jesús,et al.  A proposal on reasoning methods in fuzzy rule-based classification systems , 1999, Int. J. Approx. Reason..

[21]  Luis Magdalena,et al.  A Multiobjective Genetic Learning Process for joint Feature Selection and Granularity and Contexts Learning in Fuzzy Rule-Based Classification Systems , 2003 .

[22]  Frans Coenen,et al.  Threshold Tuning for Improved Classification Association Rule Mining , 2005, PAKDD.

[23]  DebK.,et al.  A fast and elitist multiobjective genetic algorithm , 2002 .

[24]  H. Ishibuchi,et al.  Distributed representation of fuzzy rules and its application to pattern classification , 1992 .

[25]  Antonio González Muñoz,et al.  SLAVE: a genetic learning system based on an iterative approach , 1999, IEEE Trans. Fuzzy Syst..

[26]  Hisao Ishibuchi,et al.  Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning , 2007, Int. J. Approx. Reason..

[27]  Hisao Ishibuchi,et al.  Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining , 2004, Fuzzy Sets Syst..

[28]  Hisao Ishibuchi,et al.  Voting in fuzzy rule-based systems for pattern classification problems , 1999, Fuzzy Sets Syst..

[29]  Antonio F. Gómez-Skarmeta,et al.  Accurate, transparent and compact fuzzy models by multi-objective evolutionary algorithms , 2003 .

[30]  Matthew A. Kupinski,et al.  Multiobjective Genetic Optimization of Diagnostic Classifiers with Implications for Generating ROC Curves , 1999, IEEE Trans. Medical Imaging.

[31]  Heikki Mannila,et al.  Verkamo: Fast Discovery of Association Rules , 1996, KDD 1996.

[32]  Hisao Ishibuchi,et al.  Effect of rule weights in fuzzy rule-based classification systems , 2001, IEEE Trans. Fuzzy Syst..

[33]  M. Anastasio,et al.  Multiobjective genetic optimization of diagnostic classifiers with implications for generating receiver operating characteristic curves , 1999, IEEE Transactions on Medical Imaging.

[34]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[35]  Xin Yao,et al.  DIVACE: Diverse and Accurate Ensemble Learning Algorithm , 2004, IDEAL.