Solving optimization problems using a hybrid systolic search on GPU plus CPU

In recent years, graphics processing units (GPUs) have emerged as a powerful architecture for solving a broad spectrum of applications in very short periods of time. However, most existing GPU optimization approaches do not exploit the full power available in a CPU–GPU platform. They have a tendency to leave one of them partially unused (usually the CPU) and fail to establish an accurate exchange of information that could help solve the target problem efficiently. Thus, better performance is expected from devising a hybrid CPU–GPU parallel algorithm that combines the highly parallel stream processing power of GPUs with the higher power of multi-core architectures. We have developed a hybrid methodology to efficiently solve optimization problems. We use a hybrid CPU–GPU architecture, to benefit from running it, in parallel, on both the CPU and the GPU. Our experiments over a heterogeneous set of combinatorial optimization problems with increasing dimensionality show a time gain of up to $$365\times $$365× in our proposal, while demonstrating high numerical accuracy. This work is intended to open up a new line of research that matches both architectures with new algorithms and cooperation techniques.

[1]  Karl Rupp,et al.  Solving 3D incompressible Navier-Stokes equations on hybrid CPU/GPU systems , 2014, SpringSim.

[2]  E. Alba,et al.  Towards the Design of Systolic Genetic Search , 2012, 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum.

[3]  Enrique Alba,et al.  Systolic Optimization on GPU Platforms , 2011, EUROCAST.

[4]  David Johnson,et al.  Particle Swarm Optimization on a GPU , 2012, 2012 IEEE International Conference on Electro/Information Technology.

[5]  Wayne Luk,et al.  A comparison of CPUs, GPUs, FPGAs, and massively parallel processor arrays for random number generation , 2009, FPGA '09.

[6]  Enrique Alba,et al.  The influence of grid shape and asynchronicity on cellular evolutionary algorithms , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[7]  H. T. Kung,et al.  Systolic Arrays for (VLSI). , 1978 .

[8]  Abraham Duarte,et al.  Advanced Scatter Search for the Max-Cut Problem , 2009, INFORMS J. Comput..

[9]  Nicolas Lachiche,et al.  Coarse grain parallelization of evolutionary algorithms on GPGPU cards with EASEA , 2009, GECCO.

[10]  Enrique Alba,et al.  Optimization Techniques for Solving Complex Problems , 2009 .

[11]  William B. Langdon,et al.  Large-Scale Bioinformatics Data Mining with Parallel Genetic Programming on Graphics Processing Units , 2013, Massively Parallel Evolutionary Computation on GPGPUs.

[12]  El-Ghazali Talbi,et al.  A Taxonomy of Hybrid Metaheuristics , 2002, J. Heuristics.

[13]  Sun-Yuan Kung,et al.  On supercomputing with systolic/wavefront array processors , 1984 .

[14]  Jens H. Krüger,et al.  A Survey of General‐Purpose Computation on Graphics Hardware , 2007, Eurographics.

[15]  Kenli Li,et al.  Performance Optimization Using Partitioned SpMV on GPUs and Multicore CPUs , 2015, IEEE Transactions on Computers.

[16]  Enrique Alba,et al.  Systolic neighborhood search on graphics processing units , 2014, Soft Comput..

[17]  Kalmanje Krishnakumar,et al.  Micro-Genetic Algorithms For Stationary And Non-Stationary Function Optimization , 1990, Other Conferences.

[18]  Cyril Fonlupt,et al.  Population Parallel GP on the G80 GPU , 2008, EuroGP.

[19]  Nicolas Lachiche,et al.  EASEA: specification and execution of evolutionary algorithms on GPGPU , 2011, Soft Computing.

[20]  Kama Huang,et al.  Application of Micro-Genetic Algorithm to the Design of Matched High Gain Patch Antenna with Zero-Refractive-Index Metamaterial Lens , 2010 .

[21]  Joseph M. Lancaster,et al.  Visions for application development on hybrid computing systems , 2008, Parallel Comput..

[22]  Antonio Pescapè,et al.  Genetic Algorithm Modeling with GPU Parallel Computing Technology , 2012, WIRN.

[23]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[24]  Wu-chun Feng,et al.  On the Efficacy of a Fused CPU+GPU Processor (or APU) for Parallel Computing , 2011, 2011 Symposium on Application Accelerators in High-Performance Computing.

[25]  Enrique Alba,et al.  Cellular Genetic Algorithm on Graphic Processing Units , 2010, NICSO.

[26]  Fred W. Glover,et al.  Solving large scale Max Cut problems via tabu search , 2013, J. Heuristics.

[27]  Ke Ding,et al.  Comparison of random number generators in Particle Swarm Optimization algorithm , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[28]  J. Kulpa,et al.  Time-frequency analysis using NVIDIA compute unified device architecture (CUDA) , 2009, Symposium on Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments (WILGA).

[29]  Trilochan Sastry,et al.  Easy weighted majority games , 2000, Math. Soc. Sci..

[30]  Kalyanmoy Deb,et al.  Massive Multimodality, Deception, and Genetic Algorithms , 1992, PPSN.

[31]  Matheus Nohra Haddad,et al.  A Hybrid CPU-GPU Local Search Heuristic for the Unrelated Parallel Machine Scheduling Problem , 2012, 2012 Third Workshop on Applications for Multi-Core Architecture.

[32]  J I Agulleiro,et al.  Hybrid computing: CPU+GPU co-processing and its application to tomographic reconstruction. , 2012, Ultramicroscopy.

[33]  Stephen Marshall,et al.  Convergence Criteria for Genetic Algorithms , 2000, SIAM J. Comput..

[34]  H. T. Kung Systolic array , 2003 .

[35]  Gary J. Koehler,et al.  New stopping criterion for genetic algorithms , 2000, Eur. J. Oper. Res..

[36]  Václav Snásel,et al.  Many-threaded implementation of differential evolution for the CUDA platform , 2011, GECCO '11.

[37]  El-Ghazali Talbi,et al.  Parallel hybrid evolutionary algorithms on GPU , 2010, IEEE Congress on Evolutionary Computation.

[38]  Lee W. Howes Efficient Random Number Generation and Application Using , 2007 .

[39]  S. Russel and P. Norvig,et al.  “Artificial Intelligence – A Modern Approach”, Second Edition, Pearson Education, 2003. , 2015 .

[40]  James L. Tangorra,et al.  Application of a micro-genetic algorithm for gait development on a bio-inspired robotic pectoral fin , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[41]  Martín Pedemonte,et al.  Systolic genetic search, a systolic computing-based metaheuristic , 2015, Soft Comput..

[42]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[43]  Jae-Weon Jeong,et al.  Applying micro genetic algorithm to numerical model for luminous intensity distribution of planar prism LED luminaire , 2013 .

[44]  Rafael Batres,et al.  Generation of operating procedures for a mixing tank with a micro genetic algorithm , 2013, Comput. Chem. Eng..

[45]  Jirí Jaros,et al.  Parallel Genetic Algorithm on the CUDA Architecture , 2010, EvoApplications.

[46]  Shigeyoshi Tsutsui,et al.  ACO with tabu search on a GPU for solving QAPs using move-cost adjusted thread assignment , 2011, GECCO '11.

[47]  Laureano F. Escudero,et al.  Efficient Reformulation for 0-1 Programs - Methods and Computational Results , 1993, Discret. Appl. Math..

[48]  Rong Long Wang,et al.  A genetic algorithm for subset sum problem , 2004, Neurocomputing.

[49]  H. T. Kung Let's Design Algorithms for VLSI Systems , 1979 .

[50]  Valeria Cardellini,et al.  Heterogeneous Sparse Matrix Computations on Hybrid GPU/CPU Platforms , 2013, PARCO.

[51]  El-Ghazali Talbi,et al.  GPU Computing for Parallel Local Search Metaheuristic Algorithms , 2013, IEEE Transactions on Computers.

[52]  Asim Munawar,et al.  Hybrid of genetic algorithm and local search to solve MAX-SAT problem using nVidia CUDA framework , 2009, Genetic Programming and Evolvable Machines.

[53]  Chenhan D. Yu,et al.  A CPU-GPU hybrid approach for the unsymmetric multifrontal method , 2011, Parallel Comput..

[54]  Xin Yao,et al.  Time complexity of evolutionary algorithms for combinatorial optimization: A decade of results , 2007, Int. J. Autom. Comput..

[55]  David E. Goldberg,et al.  Sizing Populations for Serial and Parallel Genetic Algorithms , 1989, ICGA.

[56]  Thomas Bäck,et al.  An evolutionary approach to combinatorial optimization problems , 1994, CSC '94.

[57]  Carlos Cotta,et al.  On Decision-Making in Strong Hybrid Evolutionary Algorithms , 1998, IEA/AIE.

[58]  Martín Pedemonte,et al.  Systolic Genetic Search for Software Engineering: The Test Suite Minimization Case , 2014, EvoApplications.

[59]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .