Massively Parallel Evolutionary Computation on GPGPUs

Evolutionary algorithms (EAs) are metaheuristics that learn from natural collective behavior and are applied to solve optimization problems in domains such as scheduling, engineering, bioinformatics, and finance. Such applications demand acceptable solutions with high-speed execution using finite computational resources. Therefore, there have been many attempts to develop platforms for running parallel EAs using multicore machines, massively parallel cluster machines, or grid computing environments. Recent advances in general-purpose computing on graphics processing units (GPGPU) have opened up this possibility for parallel EAs, and this is the first book dedicated to this exciting development. The three chapters of Part I are tutorials, representing a comprehensive introduction to the approach, explaining the characteristics of the hardware used, and presenting a representative project to develop a platform for automatic parallelization of evolutionary computing (EC) on GPGPUs. The 10 chapters in Part II focus on how to consider key EC approaches in the light of this advanced computational technique, in particular addressing generic local search, tabu search, genetic algorithms, differential evolution, swarm optimization, ant colony optimization, systolic genetic search, genetic programming, and multiobjective optimization. The 6 chapters in Part III present successful results from real-world problems in data mining, bioinformatics, drug discovery, crystallography, artificial chemistries, and sudoku. Although the parallelism of EAs is suited to the single-instruction multiple-data (SIMD)-based GPU, there are many issues to be resolved in design and implementation, and a key feature of the contributions is the practical engineering advice offered. This book will be of value to researchers, practitioners, and graduate students in the areas of evolutionary computation and scientific computing.

[1]  Weihang Zhu,et al.  Massively parallel differential evolution—pattern search optimization with graphics hardware acceleration: an investigation on bound constrained optimization problems , 2011, J. Glob. Optim..

[2]  Dmitri Yudanov,et al.  GPU-based implementation of real-time system for spiking neural networks , 2009 .

[3]  Siu-Ming Yiu,et al.  SOAP3: ultra-fast GPU-based parallel alignment tool for short reads , 2012, Bioinform..

[4]  Andrew J. Page,et al.  Framework for Task Scheduling in Heterogeneous Distributed Computing Using Genetic Algorithms , 2005, Artificial Intelligence Review.

[5]  Nikolaos V. Sahinidis,et al.  GPU-BLAST: using graphics processors to accelerate protein sequence alignment , 2010, Bioinform..

[6]  Jörn Loviscach,et al.  Evolutionary Design of BRDFs , 2003, Eurographics.

[7]  George D. Magoulas,et al.  Strategies to minimise the total run time of cyclic graph based genetic programming with GPUs , 2009, GECCO '09.

[8]  William B. Langdon.  Initial experiences of the Emerald: e-Infrastructure South GPU supercomputer , 2012 .

[9]  Raghavendra D. Prabhu,et al.  SOMGPU: An unsupervised pattern classifier on Graphical Processing Unit , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[10]  George D. Magoulas,et al.  Identifying similarities in TMBL programs with alignment to quicken their compilation for GPUs: computational intelligence on consumer games and graphics hardware , 2011, GECCO '11.

[11]  Fatos Xhafa,et al.  Genetic algorithm based schedulers for grid computing systems , 2007 .

[12]  Gang Wang,et al.  MrBayes on a Graphics Processing Unit , 2011, Bioinform..

[13]  Peter J. Bentley,et al.  Systemic Computation Using Graphics Processors , 2010, ICES.

[14]  Can Yang,et al.  GBOOST: a GPU-based tool for detecting gene-gene interactions in genome-wide case control studies , 2011, Bioinform..

[15]  Suvranu De,et al.  CUDA-based Real Time Surgery Simulation , 2008, MMVR.

[16]  Alexandros Stamatakis,et al.  RAxML-VI-HPC: maximum likelihood-based phylogenetic analyses with thousands of taxa and mixed models , 2006, Bioinform..

[17]  Weiguo Liu,et al.  Bio-sequence database scanning on a GPU , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[18]  P. S. Thiagarajan,et al.  Approximate probabilistic analysis of biopathway dynamics , 2012, Bioinform..

[19]  Václav Snásel,et al.  Many-threaded implementation of differential evolution for the CUDA platform , 2011, GECCO '11.

[20]  William B. Langdon,et al.  GP on SPMD parallel graphics hardware for mega Bioinformatics data mining , 2008, Soft Comput..

[21]  Amit Konar,et al.  Automatic Hard Clustering Using Improved Differential Evolution Algorithm , 2009 .

[22]  Shin Yoo,et al.  Evolving Human Competitive Spectra-Based Fault Localisation Techniques , 2012, SSBSE.

[23]  Chee Keong Kwoh,et al.  CBESW: Sequence Alignment on the Playstation 3 , 2008, BMC Bioinformatics.

[24]  Wolfgang Banzhaf,et al.  Deployment of CPU and GPU-based genetic programming on heterogeneous devices , 2009, GECCO '09.

[25]  Nicolas Lachiche,et al.  Fast Evaluation of GP Trees on GPGPU by Optimizing Hardware Scheduling , 2010, EuroGP.

[26]  William B. Langdon,et al.  Generalisation in genetic programming , 2011, GECCO.

[27]  Peter Nordin,et al.  A compiling genetic programming system that directly manipulates the machine-code , 1994 .

[28]  Nicolas Lachiche,et al.  Coarse grain parallelization of evolutionary algorithms on GPGPU cards with EASEA , 2009, GECCO.

[29]  Václav Snásel,et al.  Differential Evolution and Genetic Algorithms for the Linear Ordering Problem , 2009, KES.

[30]  Václav Snásel,et al.  Comparison of Heuristics for Scheduling Independent Tasks on Heterogeneous Distributed Environments , 2009, 2009 International Joint Conference on Computational Sciences and Optimization.

[31]  Ajith Abraham,et al.  A modified differential evolution for autonomous deployment and localization of sensor nodes , 2011, GECCO '11.

[32]  Jose L. Contreras-Vidal,et al.  Development of a Large-Scale Integrated Neurocognitive Architecture Part 2: Design and Architecture , 2006 .

[33]  W. Langdon,et al.  RNAnet a Map of Human Gene Expression , 2010, 1001.4263.

[34]  Andrew Lim,et al.  Designing A Hybrid Genetic Algorithm for the Linear Ordering Problem , 2003, GECCO.

[35]  Wolfgang Banzhaf,et al.  Linear genetic programming GPGPU on Microsoft’s Xbox 360 , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[36]  Cyril Fonlupt,et al.  Population Parallel GP on the G80 GPU , 2008, EuroGP.

[37]  Cyril Fonlupt,et al.  High performance genetic programming on GPU , 2009, BADS '09.

[38]  William B. Langdon,et al.  Creating and Debugging Performance CUDA C , 2012, Parallel Architectures and Bioinspired Algorithms.

[39]  J. Mitchell,et al.  Solving Linear Ordering Problems with a Combined Interior Pointtsimplex Cutting Plane Algorithm * , 1999 .

[40]  Jack J. Dongarra,et al.  Experiments with Scheduling Using Simulated Annealing in a Grid Environment , 2002, GRID.

[41]  Martín Pedemonte,et al.  PUGACE, a cellular Evolutionary Algorithm framework on GPUs , 2010, IEEE Congress on Evolutionary Computation.

[42]  P. Hall,et al.  An expression signature for p53 status in human breast cancer predicts mutation status, transcriptional effects, and patient survival. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[43]  Jan Platos,et al.  Differential Evolution for Scheduling Independent Tasks on Heterogeneous Distributed Environments , 2010 .

[44]  Renato A. Krohling,et al.  Differential evolution algorithm on the GPU with C-CUDA , 2010, IEEE Congress on Evolutionary Computation.

[45]  David Fernández-Baca,et al.  Allocating Modules to Processors in a Distributed System , 1989, IEEE Trans. Software Eng..

[46]  Giorgio Valle,et al.  CUDA compatible GPU cards as efficient hardware accelerators for Smith-Waterman sequence alignment , 2008, BMC Bioinformatics.

[47]  Jason Lawrence,et al.  Genetic programming for shader simplification , 2011, ACM Trans. Graph..

[48]  William B. Langdon,et al.  Genetic Programming in Data Mining for Drug Discovery , 2005 .

[49]  Gerhard Reinelt,et al.  A benchmark library and a comparison of heuristic methods for the linear ordering problem , 2012, Comput. Optim. Appl..

[50]  Michael P. H. Stumpf,et al.  GPU accelerated biochemical network simulation , 2011, Bioinform..

[51]  William B. Langdon.  Large Scale Bioinformatics Data Mining with Parallel Genetic Programming on Graphics Processing Units , 2010, Parallel and Distributed Computational Intelligence.

[52]  Jens H. Krüger,et al.  A Survey of General‐Purpose Computation on Graphics Hardware , 2007, Eurographics.

[53]  Man Leung Wong,et al.  Parallel multi-objective evolutionary algorithms on graphics processing units , 2009, GECCO '09.

[54]  Martín Pedemonte,et al.  Bitwise operations for GPU implementation of genetic algorithms , 2011, GECCO '11.

[55]  Fred W. Glover,et al.  An Experimental Evaluation of a Scatter Search for the Linear Ordering Problem , 2001, J. Glob. Optim..

[56]  Thomas Stützle,et al.  Search Space Analysis of the Linear Ordering Problem , 2003, EvoWorkshops.

[57]  William B. Langdon,et al.  A Many Threaded CUDA Interpreter for Genetic Programming , 2010, EuroGP.

[58]  William B. Langdon,et al.  Graphics processing units and genetic programming: an overview , 2011, Soft Comput..

[59]  R. Storn,et al.  Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces , 2004 .

[60]  William B. Langdon,et al.  A fast high quality pseudo random number generator for nVidia CUDA , 2009, GECCO '09.

[61]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[62]  Gerhard Reinelt,et al.  The Linear Ordering Problem: Exact and Heuristic Methods in Combinatorial Optimization , 2011 .

[63]  Rainer Storn,et al.  Differential evolution design of an IIR-filter , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[64]  William B. Langdon,et al.  Genetic Programming for Mining DNA Chip Data from Cancer Patients , 2004, Genetic Programming and Evolvable Machines.

[65]  John Levine,et al.  A hybrid ant algorithm for scheduling independent jobs in heterogeneous computing environments , 2004 .

[66]  William B. Langdon.  Distilling GeneChips with Genetic Programming on the Emerald GPU supercomputer , 2012 .

[67]  Jian-Zhong Li,et al.  Performance Analysis of Task Scheduling Heuristics in Grid , 2007, 2007 International Conference on Machine Learning and Cybernetics.

[68]  Mark Harman,et al.  Evolving a CUDA kernel from an nVidia template , 2010, IEEE Congress on Evolutionary Computation.

[69]  Noel Lopes,et al.  High-performance bankruptcy prediction model using Graphics Processing Units , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[70]  John D. Owens,et al.  GPU Computing , 2008, Proceedings of the IEEE.

[71]  Cyril Fonlupt,et al.  Genetic programming on graphics processing units , 2009, Genetic Programming and Evolvable Machines.

[72]  Matteo Fischetti,et al.  The Linear Ordering Problem with cumulative costs , 2008, Eur. J. Oper. Res..

[73]  Daniel A. Ashlock,et al.  Evolutionary computation for modeling and optimization , 2005 .

[74]  William B. Langdon,et al.  A SIMD Interpreter for Genetic Programming on GPU Graphics Cards , 2007, EuroGP.

[75]  Tatsuo Unemi,et al.  SBArt4 — Breeding abstract animations in realtime , 2010, IEEE Congress on Evolutionary Computation.

[76]  Zhongwen Luo,et al.  Artificial neural network computation on graphic process unit , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[77]  Asim Munawar,et al.  Hybrid of genetic algorithm and local search to solve MAX-SAT problem using nVidia CUDA framework , 2009, Genetic Programming and Evolvable Machines.

[78]  Andries P. Engelbrecht,et al.  Computational Intelligence: An Introduction , 2002 .

[79]  George D. Magoulas,et al.  TMBL kernels for CUDA GPUs compile faster using PTX: computational intelligence on consumer games and graphics hardware , 2011, GECCO '11.

[80]  Riccardo Poli,et al.  Foundations of Genetic Programming , 1999, Springer Berlin Heidelberg.

[81]  Václav Snásel,et al.  Optimization of Turbo Codes by Differential Evolution and Genetic Algorithms , 2009, 2009 Ninth International Conference on Hybrid Intelligent Systems.

[82]  William B. Langdon.  Debugging CUDA , 2011, GECCO '11.

[83]  Nicolas Lachiche,et al.  EASEA parallelization of tree-based Genetic Programming , 2010, IEEE Congress on Evolutionary Computation.

[84]  Gerhard Reinelt,et al.  The linear ordering problem: algorithms and applications , 1985 .

[85]  G.E. Moore,et al.  Cramming More Components Onto Integrated Circuits , 1998, Proceedings of the IEEE.

[86]  Michael O'Neill,et al.  Acceleration of grammatical evolution using graphics processing units: computational intelligence on consumer games and graphics hardware , 2011, GECCO.

[87]  Jirí Jaros,et al.  Parallel Genetic Algorithm on the CUDA Architecture , 2010, EvoApplications.

[88]  Václav Snásel,et al.  Evolving Schedules of Independent Tasks by Differential Evolution , 2011, Intelligent Networking, Collaborative Systems and Applications.

[89]  Krister Wolff,et al.  Evolving 3D model interpretation of images using graphics hardware , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[90]  Cole Trapnell,et al.  Optimizing data intensive GPGPU computations for DNA sequence alignment , 2009, Parallel Comput..

[91]  Debashis Ghosh,et al.  Feature selection and molecular classification of cancer using genetic programming. , 2007, Neoplasia.

[92]  T. Stützle,et al.  The Linear Ordering Problem: Instances, Search Space Analysis and Algorithms , 2004 .

[93]  Shigeyoshi Tsutsui,et al.  A Highly-Parallel TSP Solver for a GPU Computing Platform , 2010, NMA.

[94]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[95]  George R. Price,et al.  Selection and Covariance , 1970, Nature.

[96]  Wolfgang Banzhaf,et al.  Deployment of parallel linear genetic programming using GPUs on PC and video game console platforms , 2010, Genetic Programming and Evolvable Machines.

[97]  El-Ghazali Talbi,et al.  Parallel hybrid evolutionary algorithms on GPU , 2010, IEEE Congress on Evolutionary Computation.

[98]  William B. Langdon,et al.  A Survey of Spatial Defects in Homo Sapiens Affymetrix GeneChips , 2010, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[99]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[100]  David P. Anderson,et al.  An analysis of massively distributed evolutionary algorithms , 2010, IEEE Congress on Evolutionary Computation.

[101]  Mark Harman,et al.  Formal Concept Analysis on Graphics Hardware , 2011, CLA.

[102]  Yuji Sato,et al.  Acceleration experiment of genetic computations for sudoku solution on multi-core processors , 2011, GECCO '11.

[103]  Václav Snásel,et al.  Differential evolution for the linear ordering problem implemented on CUDA , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[104]  Václav Snásel,et al.  Scheduling Independent Tasks on Heterogeneous Distributed Environments by Differential Evolution , 2009, 2009 International Conference on Intelligent Networking and Collaborative Systems.

[105]  Ladislau Bölöni,et al.  A Comparison of Eleven Static Heuristics for Mapping a Class of Independent Tasks onto Heterogeneous Distributed Computing Systems , 2001, J. Parallel Distributed Comput..

[106]  A. A. Maciejewski,et al.  Heterogeneous Computing , 2002 .