Classification With Ant Colony Optimization

Ant colony optimization (ACO) can be applied to the data mining field to extract rule-based classifiers. The aim of this paper is twofold. On the one hand, we provide an overview of previous ant-based approaches to the classification task and compare them with state-of-the-art classification techniques, such as C4.5, RIPPER, and support vector machines in a benchmark study. On the other hand, a new ant-based classification technique is proposed, named AntMiner+. The key differences between the proposed AntMiner+ and previous AntMiner versions are the usage of the better performing MAX-MIN ant system, a clearly defined and augmented environment for the ants to walk through, with the inclusion of the class variable to handle multiclass problems, and the ability to include interval rules in the rule list. Furthermore, the commonly encountered problem in ACO of setting system parameters is dealt with in an automated, dynamic manner. Our benchmarking experiments show an AntMiner+ accuracy that is superior to that obtained by the other AntMiner versions, and competitive or better than the results achieved by the compared classification techniques.

[1]  Frank Harary,et al.  Graph Theory , 2016 .

[2]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[3]  Ajith Abraham,et al.  Web usage mining using artificial ant colony clustering and linear genetic programming , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[4]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.

[5]  Marco Dorigo,et al.  AntNet: Distributed Stigmergetic Control for Communications Networks , 1998, J. Artif. Intell. Res..

[6]  Brian D. Ripley,et al.  Neural Networks and Related Methods for Classification , 1994 .

[7]  David J. Hand,et al.  Pattern Detection and Discovery , 2002, Pattern Detection and Discovery.

[8]  M J Pazzani,et al.  Acceptance of Rules Generated by Machine Learning among Medical Experts , 2001, Methods of Information in Medicine.

[9]  Johan A. K. Suykens,et al.  Benchmarking state-of-the-art classification algorithms for credit scoring , 2003, J. Oper. Res. Soc..

[10]  M. Dorigo,et al.  1 Positive Feedback as a Search Strategy , 1991 .

[11]  E. Kay,et al.  Graph Theory. An Algorithmic Approach , 1975 .

[12]  Bo Liu,et al.  Density-Based Heuristic for Rule Discovery with Ant-Miner , 2002 .

[13]  Holger H. Hoos,et al.  Improving the Ant System: A Detailed Report on the MAX-MIN Ant System , 1996 .

[14]  Richard F. Hartl,et al.  Applying the ANT System to the Vehicle Routing Problem , 1999 .

[15]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[16]  Luca Maria Gambardella,et al.  Ant-Q: A Reinforcement Learning Approach to the Traveling Salesman Problem , 1995, ICML.

[17]  Johan A. K. Suykens,et al.  Benchmarking Least Squares Support Vector Machine Classifiers , 2004, Machine Learning.

[18]  Marco Dorigo,et al.  Ant-Based Clustering and Topographic Mapping , 2006, Artificial Life.

[19]  Michael Sampels,et al.  A MAX-MIN Ant System for the University Course Timetabling Problem , 2002, Ant Algorithms.

[20]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[21]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[22]  Roberto Montemanni,et al.  Ant Colony System for a Dynamic Vehicle Routing Problem , 2005, J. Comb. Optim..

[23]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[24]  Bart Baesens,et al.  Decision diagrams in machine learning: an empirical study on real-life credit-risk data , 2004, Expert Syst. Appl..

[25]  B. Bullnheimer,et al.  A NEW RANK BASED VERSION OF THE ANT SYSTEM: A COMPUTATIONAL STUDY , 1997 .

[26]  David J. Hand,et al.  Discrimination and Classification , 1982 .

[27]  Thomas Stützle,et al.  MAX-MIN Ant System , 2000, Future Gener. Comput. Syst..

[28]  Said Salhi,et al.  An ant system algorithm for the mixed vehicle routing problem with backhauls , 2004 .

[29]  Christian Blum,et al.  Beam-ACO - hybridizing ant colony optimization with beam search: an application to open shop scheduling , 2005, Comput. Oper. Res..

[30]  Marco Dorigo,et al.  Ant system for Job-shop Scheduling , 1994 .

[31]  Alex Alves Freitas,et al.  Data mining with an ant colony optimization algorithm , 2002, IEEE Trans. Evol. Comput..

[32]  W. R. Shankle,et al.  Acceptance by medical experts of rules generated by machine learning , 2001 .

[33]  Marco Dorigo,et al.  Swarm intelligence: from natural to artificial systems , 1999 .

[34]  Jonathan N. Crook,et al.  Credit Scoring and Its Applications , 2002, SIAM monographs on mathematical modeling and computation.

[35]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[36]  Hussein A. Abbass,et al.  Classification rule discovery with ant colony optimization , 2003, IEEE/WIC International Conference on Intelligent Agent Technology, 2003. IAT 2003..

[37]  Bart BaesensRudy.  Using Neural Network Rule Extraction and Decision Tables for Credit-Risk Evaluation , 2003 .

[38]  Marco Dorigo,et al.  The ant colony optimization meta-heuristic , 1999 .

[39]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[40]  Alex A. Freitas,et al.  An ant colony based system for data mining: applications to medical data , 2001 .