Parallel multi-objective Ant Programming for classification using GPUs

Classification using Ant Programming is a challenging data mining task which demands a great deal of computational resources when handling data sets of high dimensionality. This paper presents a new parallelization approach of an existing multi-objective Ant Programming model for classification, using GPUs and the NVIDIA CUDA programming model. The computational costs of the different steps of the algorithm are evaluated and it is discussed how best to parallelize them. The features of both the CPU parallel and GPU versions of the algorithm are presented. An experimental study is carried out to evaluate the performance and efficiency of the interpreter of the rules, and reports the execution times and speedups regarding variable population size, complexity of the rules mined and dimensionality of the data sets. Experiments measure the original single-threaded and the new multi-threaded CPU and GPU times with different number of GPU devices. The results are reported in terms of the number of Giga GP operations per second of the interpreter (up to 10 billion GPops/s) and the speedup achieved (up to 834x vs CPU, 212x vs 4-threaded CPU). The proposed GPU model is demonstrated to scale efficiently to larger datasets and to multiple GPU devices, which allows the expansion of its applicability to significantly more complicated data sets, previously unmanageable by the original algorithm in reasonable time.

[1]  Steven Guan,et al.  An incremental approach to genetic-algorithms-based classification , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[2]  Robin M. Weiss GPU-Accelerated Ant Colony Optimization , 2011 .

[3]  Ximing Li,et al.  MAX-MIN Ant System on GPU with CUDA , 2009, 2009 Fourth International Conference on Innovative Computing, Information and Control (ICICIC).

[4]  Sebastián Ventura,et al.  Classification rule mining using ant programming guided by grammar with multiple Pareto fronts , 2012, Soft Comput..

[5]  Tien-Tsin Wong,et al.  Evolutionary Computing on Consumer Graphics Hardware , 2007, IEEE Intelligent Systems.

[6]  Guohua Zhou,et al.  A parallel Ant Colony Optimization algorithm with GPU-acceleration based on All-In-Roulette selection , 2010, Third International Workshop on Advanced Computational Intelligence.

[7]  Alex Alves Freitas,et al.  Data mining with an ant colony optimization algorithm , 2002, IEEE Trans. Evol. Comput..

[8]  John D. Owens,et al.  GPU Computing , 2008, Proceedings of the IEEE.

[9]  Ling Chen,et al.  Parallel ant colony algorithm for mining classification rules , 2006, 2006 IEEE International Conference on Granular Computing.

[10]  S. N. Omkar,et al.  Rule extraction for classification of acoustic emission signals using Ant Colony Optimisation , 2008, Eng. Appl. Artif. Intell..

[11]  Monique Snoeck,et al.  Classification With Ant Colony Optimization , 2007, IEEE Transactions on Evolutionary Computation.

[12]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[13]  Michael Vassilakopoulos,et al.  Ant Colony Optimization and Data Mining: Techniques and Trends , 2010, 2010 International Conference on P2P, Parallel, Grid, Cloud and Internet Computing.

[14]  Cheng Wang,et al.  Parallel data mining techniques on Graphics Processing Unit with Compute Unified Device Architecture (CUDA) , 2011, The Journal of Supercomputing.

[15]  Jaume Bacardit,et al.  Speeding up the evaluation of evolutionary learning systems using GPGPUs , 2010, GECCO '10.

[16]  Martín Pedemonte,et al.  A survey on parallel ant colony optimization , 2011, Appl. Soft Comput..

[17]  Martyn Amos,et al.  Parallelization strategies for ant colony optimisation on GPUs , 2011, 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum.

[18]  Daniel Angus,et al.  Multiple objective ant colony optimisation , 2009, Swarm Intelligence.

[19]  Martyn Amos,et al.  Enhancing data parallelism for Ant Colony Optimization on GPUs , 2013, J. Parallel Distributed Comput..

[20]  Mohammad Saniee Abadeh,et al.  A fuzzy classification system based on Ant Colony Optimization for diabetes disease diagnosis , 2011, Expert Syst. Appl..

[21]  Wen-mei W. Hwu,et al.  Illinois ECE 498AL: Programming Massively Parallel Processors, Lecture 13: Reductions and their Implementation , 2009 .

[22]  S. Priyanka,et al.  Parallel Ant-Miner (PAM) on High Performance Clusters , 2010, SEMCCO.

[23]  Enrique Alba,et al.  Time analysis of standard evolutionary algorithms as software programs , 2011, 2011 11th International Conference on Intelligent Systems Design and Applications.

[24]  Tong Heng Lee,et al.  Evolutionary computing for knowledge discovery in medical diagnosis , 2003, Artif. Intell. Medicine.

[25]  Wang Jiening,et al.  Implementation of Ant Colony Algorithm Based on GPU , 2009, 2009 Sixth International Conference on Computer Graphics, Imaging and Visualization.

[26]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[27]  Kamran Zamanifar,et al.  Parallel Ant Miner 2 , 2008, ICAISC.

[28]  Enrique Alba,et al.  Parallelism and evolutionary algorithms , 2002, IEEE Trans. Evol. Comput..

[29]  Mária Lucká,et al.  Parallel Cooperative Savings Based Ant Colony Optimization - Multiple Search and Decomposition Approaches , 2006, Parallel Process. Lett..

[30]  Chun-Nan Hsu,et al.  Bayesian classification for data from the same unknown class , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[31]  Enrique Alba,et al.  Parallel Models for Genetic Algorithms , 2011 .

[32]  Wolfgang Banzhaf,et al.  Accelerating Genetic Programming through Graphics Processing Units. , 2009 .

[33]  William B. Langdon,et al.  Graphics processing units and genetic programming: an overview , 2011, Soft Comput..

[34]  Vojislav Kecman,et al.  Kernel Based Algorithms for Mining Huge Data Sets: Supervised, Semi-supervised, and Unsupervised Learning , 2006, Studies in Computational Intelligence.

[35]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[37]  Kwong-Sak Leung,et al.  Data Mining Using Grammar Based Genetic Programming and Applications , 2000 .

[38]  Milan Tuba,et al.  Comparison of different topologies for island-based multi-colony ant algorithms for the minimum weight vertex cover problem , 2010 .

[39]  Martín Pedemonte,et al.  A cellular ant colony optimisation for the generalised Steiner problem , 2010 .

[40]  Sotiris B. Kotsiantis,et al.  Machine learning: a review of classification and combining techniques , 2006, Artificial Intelligence Review.

[41]  Ken A. Hawick,et al.  Exploiting graphical processing units for data-parallel scientific applications , 2009 .

[42]  William B. Langdon,et al.  A Many Threaded CUDA Interpreter for Genetic Programming , 2010, EuroGP.

[43]  Sanjeev R. Kulkarni,et al.  Learning Pattern Classification - A Survey , 1998, IEEE Trans. Inf. Theory.

[44]  Yao Zhang,et al.  Parallel Computing Experiences with CUDA , 2008, IEEE Micro.

[45]  Sebastián Ventura,et al.  Speeding up the evaluation phase of GP classification algorithms on GPUs , 2012, Soft Comput..

[46]  Vijay P. Kumar,et al.  Analyzing Scalability of Parallel Algorithms and Architectures , 1994, J. Parallel Distributed Comput..

[47]  Andrew Lewis,et al.  A Parallel Implementation of Ant Colony Optimization , 2002, J. Parallel Distributed Comput..

[48]  Sebastián Ventura,et al.  Using Ant Programming Guided by Grammar for Building Rule-Based Classifiers , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[49]  Marc Gravel,et al.  Parallel Ant Colony Optimization on Graphics Processing Units , 2013, J. Parallel Distributed Comput..

[50]  Naga K. Govindaraju,et al.  A Survey of General‐Purpose Computation on Graphics Hardware , 2007 .

[51]  David Tarditi,et al.  Accelerator: using data parallelism to program GPUs for general-purpose uses , 2006, ASPLOS XII.

[52]  Kevin Skadron,et al.  A performance study of general-purpose applications on graphics processors using CUDA , 2008, J. Parallel Distributed Comput..

[53]  Wen-mei W. Hwu,et al.  Illinois ECE 498AL: Programming Massively Parallel Processors , 2009 .

[54]  David R. Kaeli,et al.  Exploiting Memory Access Patterns to Improve Memory Performance in Data-Parallel Architectures , 2011, IEEE Transactions on Parallel and Distributed Systems.

[55]  Weihang Zhu,et al.  Parallel ant colony for nonlinear function optimization with graphics hardware acceleration , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.