Multiobjective Optimization and Rule Learning: Subselection Algorithm or Meta-heuristic Algorithm?

A previous work explores a Multi-Objective Subset Selection algorithm, denominated the Pareto Front Elite, to induce classifiers. These classifiers are composed by a set of rules selected following Pareto dominance concepts and forming unordered classifiers. These rules are previously created by an association rule algorithm. The performance of the classifiers induced were compared with other well known rule induction algorithms using the area under the ROC curve. The area under the ROC curve (AUC) is considered a relevant criterion to deal with imbalanced data, misclassification costs and noisy data. The results show that the Pareto Front Elite algorithm is comparable to the best known techniques. In this paper we explore multi-objective meta-heuristic approach to create rules and to build the Pareto Front using the sensitivity and specificity criteria, the chosen Metaheuristic is a Greedy Randomized Adaptive Search Procedure (GRASP) with path-relinking. We perform an experimental study to compare the two algorithms: one based on a complete set of rules, and the other based on Metaheuristic Approach. In this study we analyze the classification results, through the AUC criterion, and the Pareto Front coverage produced by each algorithm.

[1]  Alain Rakotomamonjy,et al.  Optimizing Area Under Roc Curve with SVMs , 2004, ROCAI.

[2]  Rafael Martí,et al.  GRASP and Path Relinking for 2-Layer Straight Line Crossing Minimization , 1999, INFORMS J. Comput..

[3]  Lothar Thiele,et al.  A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers , 2006 .

[4]  Tom Fawcett,et al.  Analysis and Visualization of Classifier Performance: Comparison under Imprecise Class and Cost Distributions , 1997, KDD.

[5]  M. Hansen,et al.  Evaluating the quality of approximations to the non-dominated set , 1998 .

[6]  F. Glover,et al.  Handbook of Metaheuristics , 2019, International Series in Operations Research & Management Science.

[7]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[8]  A. Campbell,et al.  Progress in Artificial Intelligence , 1995, Lecture Notes in Computer Science.

[9]  Fred Glover,et al.  Tabu Search and Adaptive Memory Programming — Advances, Applications and Challenges , 1997 .

[10]  V. J. Rayward-Smith,et al.  Data mining rules using multi-objective evolutionary algorithms , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[11]  Nada Lavrac,et al.  Classification Rule Learning with APRIORI-C , 2001, EPIA.

[12]  Tom Fawcett,et al.  Using rule sets to maximize ROC performance , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[13]  Celso C. Ribeiro,et al.  Greedy Randomized Adaptive Search Procedures , 2003, Handbook of Metaheuristics.

[14]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[15]  Victor J. Rayward-Smith,et al.  Developments on a Multi-objective Metaheuristic (MOMH) Algorithm for Finding Interesting Sets of Classification Rules , 2005, EMO.

[16]  Jiawei Han,et al.  CPAR: Classification based on Predictive Association Rules , 2003, SDM.

[17]  Marco Laumanns,et al.  PISA: A Platform and Programming Language Independent Interface for Search Algorithms , 2003, EMO.

[18]  Pedro M. Domingos,et al.  Unifying Instance-Based and Rule-Based Induction , 1996 .

[19]  Peter A. Flach,et al.  ROCCER: An Algorithm for Rule Learning Based on ROC Analysis , 2005, IJCAI.

[20]  Aurora Trinidad Ramirez Pozo,et al.  Exploring Multi-objective PSO and GRASP-PR for Rule Induction , 2008, EvoCOP.

[21]  Peter A. Flach,et al.  Rule Evaluation Measures: A Unifying View , 1999, ILP.

[22]  Jian Pei,et al.  CMAR: accurate and efficient classification based on multiple class-association rules , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[23]  Peter A. Flach,et al.  Learning Decision Trees Using the Area Under the ROC Curve , 2002, ICML.

[24]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[25]  Wynne Hsu,et al.  Integrating Classification and Association Rule Mining , 1998, KDD.

[26]  Peter Clark,et al.  Rule Induction with CN2: Some Recent Improvements , 1991, EWSL.

[27]  Aurora Trinidad Ramirez Pozo,et al.  Optimization of the AUC Criterion for Rule Subset Selection , 2007, Seventh International Conference on Intelligent Systems Design and Applications (ISDA 2007).

[28]  Michèle Sebag,et al.  ROC-Based Evolutionary Learning: Application to Medical Data Mining , 2003, Artificial Evolution.

[29]  W. J. Conover,et al.  Practical Nonparametric Statistics , 1972 .

[30]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[31]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[32]  Yaochu Jin,et al.  Multi-Objective Machine Learning , 2006, Studies in Computational Intelligence.

[33]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[34]  Mauricio G. C. Resende,et al.  Grasp: An Annotated Bibliography , 2002 .

[35]  Gustavo E. A. P. A. Batista,et al.  A Comparison of Methods for Rule Subset Selection Applied to Associative Classification , 2006 .

[36]  Mauricio G. C. Resende,et al.  Greedy Randomized Adaptive Search Procedures , 1995, J. Glob. Optim..

[37]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[38]  Yoram Singer,et al.  A simple, fast, and effective rule learner , 1999, AAAI 1999.

[39]  Jeffery L. Kennington,et al.  Interfaces in Computer Science and Operations Research , 1997 .

[40]  Pedro M. Domingos,et al.  Tree Induction for Probability-Based Ranking , 2003, Machine Learning.

[41]  M. Ehrgott.  Approximation algorithms for combinatorial multicriteria optimization problems , 2000 .

[42]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[43]  Ramakrishnan Srikant,et al.  Fast algorithms for mining association rules , 1998, VLDB 1998.