Multiobjective Classification Rule Mining

In this chapter, we discuss the application of evolutionary multiobjective optimization (EMO) to association rule mining. Especially, we focus our attention on classification rule mining in a continuous feature space where the antecedent and consequent parts of each rule are an interval vector and a class label, respectively. First we explain evolutionary multiobjective classification rule mining techniques. Those techniques are roughly categorized into two approaches. In one approach, each classification rule is handled as an individual. An EMO algorithm is used to search for Pareto-optimal rules with respect to some rule evaluation criteria such as support and confidence. In the other approach, each rule set is handled as an individual. An EMO algorithm is used to search for Pareto-optimal rule sets with respect to some rule set evaluation criteria such as accuracy and complexity. Next we explain evolutionary multiobjective rule selection as a post-processing procedure in classification rule mining. Pareto-optimal rule sets are found from a large number of candidate classification rules, which are extracted from a database using an association rule mining technique. Then we examine the effectiveness of evolutionary multiobjective rule selection through computational experiments on some benchmark classification problems. Finally we examine the use of Pareto-optimal and near Pareto-optimal rules as candidate rules in evolutionary multiobjective rule selection.

[1]  Yaochu Jin,et al.  Multi-Objective Machine Learning , 2006, Studies in Computational Intelligence.

[2]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[3]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[4]  Sankar K. Pal,et al.  Data mining in soft computing framework: a survey , 2002, IEEE Trans. Neural Networks.

[5]  Hisao Ishibuchi,et al.  Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining , 2004, Fuzzy Sets Syst..

[6]  Francisco Herrera,et al.  Stratification for scaling up evolutionary prototype selection , 2005, Pattern Recognit. Lett..

[7]  Malcolm I. Heywood,et al.  Towards Efficient Training on Large Datasets for Genetic Programming , 2004, Canadian AI.

[8]  Stefan Mutter,et al.  Using Classification to Evaluate the Output of Confidence-Based Association Rule Mining , 2004, Australian Conference on Artificial Intelligence.

[9]  Sankar K. Pal,et al.  Web mining in soft computing framework: relevance, state of the art and future directions , 2002, IEEE Trans. Neural Networks.

[10]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[11]  J. Casillas.  Interpretability issues in fuzzy modeling , 2003 .

[12]  Tapio Elomaa,et al.  General and Efficient Multisplitting of Numerical Attributes , 1999, Machine Learning.

[13]  Hisao Ishibuchi,et al.  Three-objective genetics-based machine learning for linguistic rule extraction , 2001, Inf. Sci..

[14]  Heikki Mannila,et al.  Fast Discovery of Association Rules , 1996, Advances in Knowledge Discovery and Data Mining.

[15]  Kim-Fung Man,et al.  Agent-based evolutionary approach for interpretable rule-based knowledge extraction , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[16]  Bhabesh Nath,et al.  Multi-objective rule mining using genetic algorithms , 2004, Inf. Sci..

[17]  V. J. Rayward-Smith,et al.  Data mining rules using multi-objective evolutionary algorithms , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[18]  M. Anastasio,et al.  Multiobjective genetic optimization of diagnostic classifiers with implications for generating receiver operating characteristic curves , 1999, IEEE Transactions on Medical Imaging.

[19]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[20]  Frans Coenen,et al.  Threshold Tuning for Improved Classification Association Rule Mining , 2005, PAKDD.

[21]  Jian Pei,et al.  CMAR: accurate and efficient classification based on multiple class-association rules , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[22]  Hisao Ishibuchi,et al.  Classification and modeling with linguistic information granules - advanced approaches to linguistic data mining , 2004, Advanced information processing.

[23]  Xavier Llorà,et al.  Bounding the Effect of Noise in Multiobjective Learning Classifier Systems , 2003, Evolutionary Computation.

[24]  Mehmet Kaya,et al.  Multi-objective genetic algorithm based approaches for mining optimized fuzzy association rules , 2006, Soft Comput..

[25]  Kim-Fung Man,et al.  Multi-objective hierarchical genetic algorithm for interpretable fuzzy rule-based knowledge extraction , 2005, Fuzzy Sets Syst..

[26]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[27]  Peter I. Cowling,et al.  Improving rule sorting, predictive accuracy and training time in associative classification , 2006, Expert Syst. Appl..

[28]  Roberto J. Bayardo,et al.  Mining the most interesting rules , 1999, KDD '99.

[29]  Francisco Herrera,et al.  On the combination of evolutionary algorithms and stratified strategies for training set selection in data mining , 2006, Appl. Soft Comput..

[30]  Dr. Alex A. Freitas.  Data Mining and Knowledge Discovery with Evolutionary Algorithms , 2002, Natural Computing Series.

[31]  Hisao Ishibuchi,et al.  Evolutionary Multiobjective Knowledge Extraction for High-Dimensional Pattern Classification Problems , 2004, PPSN.

[32]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[33]  Tong Heng Lee,et al.  A distributed evolutionary classifier for knowledge discovery in data mining , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[34]  Matthew A. Kupinski,et al.  Multiobjective Genetic Optimization of Diagnostic Classifiers with Implications for Generating ROC Curves , 1999, IEEE Trans. Medical Imaging.

[35]  Frans Coenen,et al.  Obtaining best parameter values for accurate classification , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[36]  Beatriz de la Iglesia,et al.  Rule Induction Using Multi-Objective Metaheuristics: Encouraging Rule Diversity , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[37]  Hisao Ishibuchi,et al.  Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems , 1997, Fuzzy Sets Syst..

[38]  Victor J. Rayward-Smith,et al.  The application and effectiveness of a multi-objective metaheuristic algorithm for partial classification , 2006, Eur. J. Oper. Res..

[39]  Chaochang Chiu,et al.  A constraint-based genetic algorithm approach for mining classification rules , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[40]  Wynne Hsu,et al.  Integrating Classification and Association Rule Mining , 1998, KDD.

[41]  Hisao Ishibuchi,et al.  Accuracy-Complexity Tradeoff Analysis in Data Mining by Multiobjective Genetic Rule Selection , 2006 .

[42]  Victor J. Rayward-Smith,et al.  Developments on a Multi-objective Metaheuristic (MOMH) Algorithm for Finding Interesting Sets of Classification Rules , 2005, EMO.

[43]  Hisao Ishibuchi,et al.  Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning , 2007, Int. J. Approx. Reason..