OpenABLext: An automatic code generation framework for agent‐based simulations on CPU‐GPU‐FPGA heterogeneous platforms

The execution of agent‐based simulations (ABSs) on hardware accelerator devices such as graphics processing units (GPUs) has been shown to offer great performance potentials. However, in heterogeneous hardware environments, it can become increasingly difficult to find viable partitions of the simulation and provide implementations for different hardware devices. To automate this process, we present OpenABLext, an extension to OpenABL, a model specification language for ABSs. By providing a device‐aware OpenCL backend, OpenABLext enables the co‐execution of ABS on heterogeneous hardware platforms consisting of central processing units, GPUs, and field programmable gate arrays (FPGAs). We present a novel online dispatching method that efficiently profiles partitions of the simulation during run‐time to optimize the hardware assignment while using the profiling results to advance the simulation itself. In addition, OpenABLext features automated conflict resolution based on user‐specified rules, supports graph‐based simulation spaces, and utilizes an efficient neighbor search algorithm. We show the improved performance of OpenABLext and demonstrate the potential of FPGAs in the context of ABS. We illustrate how co‐execution can be used to further lower execution times. OpenABLext can be seen as an enabler to tap the computing power of heterogeneous hardware platforms for ABS.

[1]  Wentong Cai,et al.  Advancing Automatic Code Generation for Agent-Based Simulations on Heterogeneous Hardware , 2019, Euro-Par Workshops.

[2]  Jeff Larkin,et al.  Parallel programming with OpenACC , 2017 .

[3]  Gennaro Cordasco,et al.  A Framework for Distributing Agent-Based Simulations , 2011, Euro-Par Workshops.

[4]  Wentong Cai,et al.  A Survey on Agent-based Simulation Using Hardware Accelerators , 2018, ACM Comput. Surv..

[5]  Joshua M. Epstein,et al.  Artificial societies and generative social science , 2006, Artificial Life and Robotics.

[6]  Sean Luke,et al.  A pheromone-based utility model for collaborative foraging , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[7]  Emanuel Popovici,et al.  Power analysis of sorting algorithms on FPGA using OpenCL , 2018, 2018 29th Irish Signals and Systems Conference (ISSC).

[8]  Laxmi N. Bhuyan,et al.  A dynamic self-scheduling scheme for heterogeneous multiprocessor architectures , 2013, TACO.

[9]  Roshan M. D'Souza,et al.  A Framework for Megascale Agent Based Model Simulations on Graphics Processing Units , 2008, J. Artif. Soc. Soc. Simul..

[10]  Mariam Kiran,et al.  FLAME: simulating large populations of agents on parallel hardware architectures , 2010, AAMAS.

[11]  Ben H. H. Juurlink,et al.  OpenABL: A Domain-Specific Language for Parallel and Distributed Agent-Based Simulations , 2018, Euro-Par.

[12]  Alexey Lastovetsky,et al.  A Novel Data-Partitioning Algorithm for Performance Optimization of Data-Parallel Applications on Heterogeneous HPC Platforms , 2018, IEEE Transactions on Parallel and Distributed Systems.

[13]  Werner Dubitzky,et al.  Large-Scale Computing Techniques for Complex System Simulations , 2011 .

[14]  Wentong Cai,et al.  Evaluation of Conflict Resolution Methods for Agent-Based Simulations on the GPU , 2018, SIGSIM-PADS.

[15]  Pei Li,et al.  Automatic OpenCL Code Generation for Multi-device Heterogeneous Architectures , 2015, 2015 44th International Conference on Parallel Processing.

[16]  Michael Lees,et al.  Efficient graph-based dynamic load-balancing for parallel large-scale agent-based traffic simulation , 2014, Proceedings of the Winter Simulation Conference 2014.

[17]  Jungwon Kim,et al.  OpenACC to FPGA: A Framework for Directive-Based High-Performance Reconfigurable Computing , 2016, 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[18]  Eric Darve,et al.  Liszt: A domain specific language for building portable mesh-based PDE solvers , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[19]  Omer Khan,et al.  HeteroMap: A Runtime Performance Predictor for Efficient Processing of Graph Analytics on Heterogeneous Multi-Accelerators , 2019, 2019 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[20]  Sean Luke,et al.  MASON: A Multiagent Simulation Environment , 2005, Simul..

[21]  Ryan Kastner,et al.  Parallel Programming for FPGAs , 2018, ArXiv.

[22]  Paul Richmond Resolving Conflicts between Multiple Competing Agents in Parallel Simulations , 2014, Euro-Par Workshops.

[23]  Kunle Olukotun,et al.  Delite , 2014, ACM Trans. Embed. Comput. Syst..

[24]  Rafael Asenjo,et al.  Heterogeneous parallel_for Template for CPU–GPU Chips , 2018, International Journal of Parallel Programming.

[25]  R. Govindarajan,et al.  Fluidic Kernels: Cooperative Execution of OpenCL Programs on Multiple Heterogeneous Devices , 2014, CGO '14.

[26]  Beau Johnston,et al.  OpenCL Performance Prediction using Architecture-Independent Features , 2018, 2018 International Conference on High Performance Computing & Simulation (HPCS).

[27]  Joshua M. Epstein,et al.  Growing Artificial Societies: Social Science from the Bottom Up , 1996 .

[28]  Steve C. Maddock,et al.  A Standardised Benchmark for Assessing the Performance of Fixed Radius Near Neighbours , 2016, Euro-Par Workshops.

[29]  Zainalabedin Navabi,et al.  VHDL: Analysis and Modeling of Digital Systems , 1992 .

[30]  Daniela M. Romano,et al.  High performance cellular level agent-based simulation with FLAME for the GPU , 2010, Briefings Bioinform..

[31]  Diana Göhringer,et al.  Automatic Mapping for OpenCL-Programs on CPU/GPU Heterogeneous Platforms , 2018, ICCS.

[32]  Nelson Minar,et al.  The Swarm Simulation System: A Toolkit for Building Multi-Agent Simulations , 1996 .

[33]  Michael J. North,et al.  Experiences creating three implementations of the repast agent modeling toolkit , 2006, TOMC.

[34]  Michael F. P. O'Boyle,et al.  Portable mapping of data parallel programs to OpenCL for heterogeneous systems , 2013, Proceedings of the 2013 IEEE/ACM International Symposium on Code Generation and Optimization (CGO).

[35]  Laurent Philippe,et al.  MCMAS: A Toolkit to Benefit from Many-Core Architecure in Agent-Based Simulation , 2013, Euro-Par Workshops.

[36]  Wu-chun Feng,et al.  Bridging the FPGA programmability-portability Gap via automatic OpenCL code generation and tuning , 2016, 2016 IEEE 27th International Conference on Application-specific Systems, Architectures and Processors (ASAP).

[37]  Yves Robert,et al.  Matrix Multiplication on Heterogeneous Platforms , 2001, IEEE Trans. Parallel Distributed Syst..

[38]  Denis Barthou,et al.  Adaptive Partitioning for Iterated Sequences of Irregular OpenCL Kernels , 2018, 2018 30th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD).

[39]  Brian Vinter,et al.  A framework for general sparse matrix-matrix multiplication on GPUs and heterogeneous processors , 2015, J. Parallel Distributed Comput..

[40]  Efficient Neighbor Searching for Agent-Based Simulation on GPU , 2014, 2014 IEEE/ACM 18th International Symposium on Distributed Simulation and Real Time Applications.

[41]  Norman I. Badler,et al.  Modeling Crowd and Trained Leader Behavior during Building Evacuation , 2006, IEEE Computer Graphics and Applications.

[42]  Eduard Ayguadé,et al.  Auto-tuned OpenCL kernel co-execution in OmpSs for heterogeneous systems , 2019, J. Parallel Distributed Comput..

[43]  Wu-chun Feng,et al.  Automatic Command Queue Scheduling for Task-Parallel Workloads in OpenCL , 2015, 2015 IEEE International Conference on Cluster Computing.

[44]  Wentong Cai,et al.  Exploring Execution Schemes for Agent-Based Traffic Simulation on Heterogeneous Hardware , 2018, 2018 IEEE/ACM 22nd International Symposium on Distributed Simulation and Real Time Applications (DS-RT).

[45]  Michael J. North,et al.  Tutorial on agent-based modelling and simulation , 2005, Proceedings of the Winter Simulation Conference, 2005..

[46]  Wei Chen,et al.  Research Challenges in Modeling and Simulation for Engineering Complex Systems , 2017, Simulation Foundations, Methods and Applications.

[47]  José Luis Bosque,et al.  Cooperative CPU, GPU, and FPGA heterogeneous execution with EngineCL , 2019, The Journal of Supercomputing.

[48]  Johanna Rohde,et al.  SOCAO: Source-to-Source OpenCL Compiler for Intel-Altera FPGAs , 2017 .