NMEEF-SD: Non-dominated Multiobjective Evolutionary Algorithm for Extracting Fuzzy Rules in Subgroup Discovery

A non-dominated multiobjective evolutionary algorithm for extracting fuzzy rules in subgroup discovery (NMEEF-SD) is described and analyzed in this paper. This algorithm, which is based on the hybridization between fuzzy logic and genetic algorithms, deals with subgroup-discovery problems in order to extract novel and interpretable fuzzy rules of interest, and the evolutionary fuzzy system NMEEF-SD is based on the well-known Non-dominated Sorting Genetic Algorithm II (NSGA-II) model but is oriented toward the subgroup-discovery task using specific operators to promote the extraction of interpretable and high-quality subgroup-discovery rules. The proposal includes different mechanisms to improve diversity in the population and permits the use of different combinations of quality measures in the evolutionary process. An elaborate experimental study, which was reinforced by the use of nonparametric tests, was performed to verify the validity of the proposal, and the proposal was compared with other subgroup discovery methods. The results show that NMEEF-SD obtains the best results among several algorithms studied.

[1]  Nada Lavrac,et al.  CSM-SD: Methodology for contrast set mining through subgroup discovery , 2009, J. Biomed. Informatics.

[2]  María José del Jesús,et al.  Evolutionary algorithms for subgroup discovery in e-learning: A practical application using Moodle data , 2009, Expert Syst. Appl..

[3]  Branko Kavsek,et al.  APRIORI-SD: ADAPTING ASSOCIATION RULE LEARNING TO SUBGROUP DISCOVERY , 2006, IDA.

[4]  R. Wilsnack.  Information Control , 1980 .

[5]  Francisco Herrera,et al.  Subgroup discover in large size data sets preprocessed using stratified instance selection for increasing the presence of minority classes , 2008, Pattern Recognit. Lett..

[6]  Ignacio Rojas,et al.  Guest Editorial Genetic Fuzzy Systems: What's Next? An Introduction to the Special Section , 2007, IEEE Trans. Fuzzy Syst..

[7]  Nada Lavrac,et al.  Expert-Guided Subgroup Discovery: Methodology and Application , 2011, J. Artif. Intell. Res..

[8]  Magne Setnes,et al.  GA-fuzzy modeling and classification: complexity and performance , 2000, IEEE Trans. Fuzzy Syst..

[9]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[10]  Sam Kwong,et al.  Genetic-fuzzy rule mining approach and evaluation of feature selection techniques for anomaly intrusion detection , 2007, Pattern Recognition.

[11]  Tzung-Pei Hong,et al.  A Genetic-Fuzzy Mining Approach for Items with Multiple Minimum Supports , 2007, FUZZ-IEEE.

[12]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[13]  Frank Puppe,et al.  SD-Map - A Fast Algorithm for Exhaustive Subgroup Discovery , 2006, PKDD.

[14]  María José del Jesús,et al.  Special Issue on Genetic Fuzzy Systems and the Interpretability-Accuracy Trade-off , 2007, Int. J. Approx. Reason..

[15]  Nada Lavrac,et al.  Classification Rule Learning with APRIORI-C , 2001, EPIA.

[16]  M. Kantardzic,et al.  The Use of Emerging Patterns in the Analysis of Gene Expression Profiles for the Diagnosis and Understanding of Diseases , 2005 .

[17]  Francisco Herrera,et al.  A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability , 2009, Soft Comput..

[18]  Lotfi A. Zadeh,et al.  The Concepts of a Linguistic Variable and its Application to Approximate Reasoning , 1975 .

[19]  David B. Fogel,et al.  An Introduction to Evolutionary Computation , 2022 .

[20]  Kwong-Sak Leung,et al.  Data Mining Using Grammar Based Genetic Programming and Applications , 2000 .

[21]  Reda Alhajj,et al.  Multi-objective genetic algorithms based automated clustering for fuzzy association rules mining , 2008, Journal of Intelligent Information Systems.

[22]  Stefan Wrobel,et al.  An Algorithm for Multi-relational Discovery of Subgroups , 1997, PKDD.

[23]  A. G. Pipe,et al.  Local identification of prototypes for genetic learning of accurate TSK fuzzy rule-based systems: Research Articles , 2007 .

[24]  Tzung-Pei Hong,et al.  An improved approach to find membership functions and multiple minimum supports in fuzzy data mining , 2009, Expert Syst. Appl..

[25]  M. El-Sharkawi,et al.  Introduction to Evolutionary Computation , 2008 .

[26]  Peter A. Flach,et al.  Rule Evaluation Measures: A Unifying View , 1999, ILP.

[27]  Tomasz Imielinski,et al.  Mining association rules between sets of items in large databases , 1993, SIGMOD Conference.

[28]  DebK.,et al.  A fast and elitist multiobjective genetic algorithm , 2002 .

[29]  Witold Pedrycz.  Fuzzy Modelling: Paradigms and Practice , 2011 .

[30]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[31]  Willi Klösgen,et al.  Explora: A Multipattern and Multistrategy Discovery Assistant , 1996, Advances in Knowledge Discovery and Data Mining.

[32]  Anthony G. Pipe,et al.  Introduction: Genetic fuzzy systems , 2007, Int. J. Intell. Syst..

[33]  Geoffrey I. Webb,et al.  Supervised Descriptive Rule Discovery: A Unifying Survey of Contrast Set, Emerging Pattern and Subgroup Mining , 2009, J. Mach. Learn. Res..

[34]  Jorge Casillas,et al.  Fuzzy-XCS: A Michigan Genetic Fuzzy System , 2007, IEEE Transactions on Fuzzy Systems.

[35]  Ralf Der,et al.  Higher Coordination With Less Control—A Result of Information Maximization in the Sensorimotor Loop , 2009, Adapt. Behav..

[36]  Jian Pei,et al.  Mining frequent patterns without candidate generation , 2000, SIGMOD '00.

[37]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[38]  María José del Jesús,et al.  MOGUL: A methodology to obtain genetic fuzzy rule-based systems under the iterative rule learning approach , 1999, Int. J. Intell. Syst..

[39]  Witold Pedrycz.  Fuzzy Modelling: Paradigms and Practices , 1996 .

[40]  María José del Jesús,et al.  Multiobjective Genetic Algorithm for Extracting Subgroup Discovery Fuzzy Rules , 2007, 2007 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making.

[41]  Jorge Casillas,et al.  Special issue on “Genetic Fuzzy Systems: Recent Developments and Future Directions” , 2009, Soft Comput..

[42]  Marco Laumanns,et al.  SPEA2: Improving the Strength Pareto Evolutionary Algorithm For Multiobjective Optimization , 2002 .

[43]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[44]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic and Evolutionary Computation) , 2006 .

[45]  Francisco Herrera,et al.  Learning the membership function contexts for mining fuzzy association rules by using genetic algorithms , 2009, Fuzzy Sets Syst..

[46]  Tzung-Pei Hong,et al.  A GA-based Fuzzy Mining Approach to Achieve a Trade-off Between Number of Rules and Suitability of Membership Functions , 2006, Soft Comput..

[47]  Francisco Herrera,et al.  Genetic Fuzzy Systems - Evolutionary Tuning and Learning of Fuzzy Knowledge Bases , 2002, Advances in Fuzzy Systems - Applications and Theory.

[48]  Peter A. Flach,et al.  Rule induction for subgroup discovery with CN2-SD , 2002 .

[49]  A Abu-Hanna,et al.  A subgroup discovery approach for scrutinizing blood glucose management guidelines by the identification of hyperglycemia determinants in ICU patients. , 2008, Methods of information in medicine.

[50]  Guo-xia Dong.  1 The Use of Emerging Patterns in the Analysis of Gene Expression Profiles for the Diagnosis and Understanding of Diseases , 2003 .

[51]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[52]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[53]  Heikki Mannila,et al.  Fast Discovery of Association Rules , 1996, Advances in Knowledge Discovery and Data Mining.

[54]  David E. Goldberg,et al.  Genetic Algorithms, Tournament Selection, and the Effects of Noise , 1995, Complex Syst..

[55]  Hisao Ishibuchi,et al.  Multiobjective Genetic Fuzzy Systems: Review and Future Research Directions , 2007, 2007 IEEE International Fuzzy Systems Conference.

[56]  Francisco Herrera,et al.  Genetic fuzzy systems: taxonomy, current research trends and prospects , 2008, Evol. Intell..

[57]  Peter A. Flach,et al.  Subgroup Discovery with CN2-SD , 2004, J. Mach. Learn. Res..

[58]  María José del Jesús,et al.  Multiobjective Evolutionary Induction of Subgroup Discovery Fuzzy Rules: A Case Study in Marketing , 2006, ICDM.

[59]  Francisco Herrera,et al.  A three-stage evolutionary process for learning descriptive and approximate fuzzy-logic-controller knowledge bases from examples , 1997, Int. J. Approx. Reason..

[60]  Eyke Hüllermeier,et al.  Fuzzy methods in machine learning and data mining: Status and prospects , 2005, Fuzzy Sets Syst..

[61]  R. Iman,et al.  Approximations of the critical region of the fbietkan statistic , 1980 .

[62]  Vipin Kumar,et al.  Introduction to Data Mining, (First Edition) , 2005 .

[63]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[64]  María José del Jesús,et al.  KEEL: a software tool to assess evolutionary algorithms for data mining problems , 2008, Soft Comput..

[65]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[66]  María José del Jesús,et al.  Evolutionary Fuzzy Rule Induction Process for Subgroup Discovery: A Case Study in Marketing , 2007, IEEE Transactions on Fuzzy Systems.

[67]  Hisao Ishibuchi,et al.  Classification and modeling with linguistic information granules - advanced approaches to linguistic data mining , 2004, Advanced information processing.

[68]  Mehmet Kaya,et al.  Multi-objective genetic algorithm based approaches for mining optimized fuzzy association rules , 2006, Soft Comput..

[69]  Stephen D. Bay,et al.  Detecting Group Differences: Mining Contrast Sets , 2001, Data Mining and Knowledge Discovery.

[70]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[71]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[72]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[73]  Dr. Rainer Palm,et al.  Model Based Fuzzy Control , 1997, Springer Berlin Heidelberg.

[74]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[75]  Nada Lavrac,et al.  Propositionalization-based relational subgroup discovery with RSD , 2006, Machine Learning.

[76]  J. Zurada,et al.  NEW GENERATION OF DATA MINING APPLICATIONS , 2003 .

[77]  Ludmila I. Kuncheva,et al.  Fuzzy Classifier Design , 2000, Studies in Fuzziness and Soft Computing.