A New Sequential Covering Strategy for Inducing Classification Rules With Ant Colony Algorithms

Ant colony optimization (ACO) algorithms have been successfully applied to discover a list of classification rules. In general, these algorithms follow a sequential covering strategy, where a single rule is discovered at each iteration of the algorithm in order to build a list of rules. The sequential covering strategy has the drawback of not coping with the problem of rule interaction, i.e., the outcome of a rule affects the rules that can be discovered subsequently since the search space is modified due to the removal of examples covered by previous rules. This paper proposes a new sequential covering strategy for ACO classification algorithms to mitigate the problem of rule interaction, where the order of the rules is implicitly encoded as pheromone values and the search is guided by the quality of a candidate list of rules. Our experiments using 18 publicly available data sets show that the predictive accuracy obtained by a new ACO classification algorithm implementing the proposed sequential covering strategy is statistically significantly higher than the predictive accuracy of state-of-the-art rule induction classification algorithms.

[1]  Luca Maria Gambardella,et al.  Ant Algorithms for Discrete Optimization , 1999, Artificial Life.

[2]  Alex Alves Freitas,et al.  Multiple pheromone types and other extensions to the Ant-Miner classification rule discovery algorithm , 2011, Swarm Intelligence.

[3]  Ian H. Witten,et al.  Data Mining: Practical Machine Learning Tools and Techniques, 3/E , 2014 .

[4]  Alex A. Freitas,et al.  A hybrid PSO/ACO algorithm for discovering classification rules in data mining , 2008 .

[5]  José Ranilla,et al.  A heuristic for learning decision trees and pruning them into classification rules , 2003, AI Commun..

[6]  Marco Dorigo,et al.  The ant colony optimization meta-heuristic , 1999 .

[7]  Thomas Stützle,et al.  Improvements on the Ant-System: Introducing the MAX-MIN Ant System , 1997, ICANNGA.

[8]  Joachim Diederich,et al.  Rule Extraction from Support Vector Machines , 2008, Studies in Computational Intelligence.

[9]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[10]  Lashon B. Booker,et al.  Intelligent Behavior as an Adaptation to the Task Environment , 1982 .

[11]  Khalid M. Salama,et al.  Extensions to the Ant-Miner Classification Rule Discovery Algorithm , 2010, ANTS Conference.

[12]  Alex Alves Freitas,et al.  On the Importance of Comprehensible Classification Models for Protein Function Prediction , 2010, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[13]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[14]  D.E. Goldberg,et al.  Classifier Systems and Genetic Algorithms , 1989, Artif. Intell..

[15]  Marco Dorigo.  Ant colony optimization , 2004, Scholarpedia.

[16]  J. Ross Quinlan,et al.  Improved Use of Continuous Attributes in C4.5 , 1996, J. Artif. Intell. Res..

[17]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[18]  Gilles Venturini,et al.  SIA: A Supervised Inductive Algorithm with Genetic Search for Learning Attributes based Concepts , 1993, ECML.

[19]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[20]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery: An Overview , 1996, Advances in Knowledge Discovery and Data Mining.

[21]  Stephen F. Smith,et al.  Flexible Learning of Problem Solving Heuristics Through Adaptive Search , 1983, IJCAI.

[22]  Thomas Stützle,et al.  MAX-MIN Ant System , 2000, Future Gener. Comput. Syst..

[23]  Alex A. Freitas,et al.  An ant colony based system for data mining: applications to medical data , 2001 .

[24]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[25]  Glenn Fung,et al.  Rule extraction from linear support vector machines , 2005, KDD '05.

[26]  Jörg Rech,et al.  Knowledge Discovery in Databases , 2001, Künstliche Intell..

[27]  D. Hosmer,et al.  Applied Logistic Regression , 1991 .

[28]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[29]  David J. Groggel,et al.  Practical Nonparametric Statistics , 2000, Technometrics.

[30]  Johan A. K. Suykens,et al.  Benchmarking state-of-the-art classification algorithms for credit scoring , 2003, J. Oper. Res. Soc..

[31]  Bo Liu,et al.  Density-Based Heuristic for Rule Discovery with Ant-Miner , 2002 .

[32]  Alex Alves Freitas,et al.  Data mining with an ant colony optimization algorithm , 2002, IEEE Trans. Evol. Comput..

[33]  Monique Snoeck,et al.  Classification With Ant Colony Optimization , 2007, IEEE Transactions on Evolutionary Computation.

[34]  S. Shott,et al.  Nonparametric Statistics , 2018, The Encyclopedia of Archaeological Sciences.

[35]  Ricardo Vilalta,et al.  Metalearning - Applications to Data Mining , 2008, Cognitive Technologies.

[36]  Bart Baesens,et al.  An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models , 2011, Decis. Support Syst..

[37]  Ian H. Witten,et al.  Generating Accurate Rule Sets Without Global Optimization , 1998, ICML.

[38]  Alex Alves Freitas,et al.  A New Classification-Rule Pruning Procedure for an Ant Colony Algorithm , 2005, Artificial Evolution.

[39]  Alex Alves Freitas,et al.  Handling continuous attributes in Ant Colony Classification algorithms , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[40]  Vasant Dhar,et al.  Discovering Interesting Patterns for Investment Decision Making with GLOWER ☹—A Genetic Learner Overlaid with Entropy Reduction , 2000, Data Mining and Knowledge Discovery.

[41]  Usama M. Fayyad,et al.  On the Handling of Continuous-Valued Attributes in Decision Tree Generation , 1992, Machine Learning.

[42]  Alex Alves Freitas,et al.  cAnt-Miner: An Ant Colony Classification Algorithm to Cope with Continuous Attributes , 2008, ANTS Conference.

[43]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[44]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[45]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[46]  Bart Baesens,et al.  Editorial survey: swarm intelligence for data mining , 2010, Machine Learning.

[47]  Hussein A. Abbass,et al.  Classification rule discovery with ant colony optimization , 2003, IEEE/WIC International Conference on Intelligent Agent Technology, 2003. IAT 2003..

[48]  William Frawley,et al.  Knowledge Discovery in Databases , 1991 .

[49]  Dr. Alex A. Freitas.  Data Mining and Knowledge Discovery with Evolutionary Algorithms , 2002, Natural Computing Series.

[50]  Thomas Stützle,et al.  A Racing Algorithm for Configuring Metaheuristics , 2002, GECCO.