Dataflow coordination of data-parallel tasks via MPI 3.0

Scientific applications are often complex collections of many large-scale tasks. Mature tools exist for describing task-parallel workflows consisting of serial tasks, and a variety of tools exist for programming a single data-parallel operation. However, few tools cover the intersection of these two models. In this work, we extend the load balancing library ADLB to support parallel tasks. We demonstrate how applications can easily be composed of parallel tasks using Swift dataflow scripts, which are compiled to ADLB programs with performance comparable to hand-coded equivalents. By combining this framework with data-parallel analysis libraries, we are able to dynamically execute many instances of a parallel data analysis application in support of a parameter exploration workload.

[1]  Daniel S. Katz,et al.  Swift/T: scalable data flow programming for many-task applications , 2013, PPoPP '13.

[2]  P. Fischer,et al.  On the numerical simulation of thermal striping in the upper plenum of a fast reactor , 2010 .

[3]  Robert B. Ross,et al.  Versatile Communication Algorithms for Data Analysis , 2012, EuroMPI.

[4]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[5]  Daniel S. Katz,et al.  Turbine: a distributed-memory dataflow engine for extreme-scale many-task applications , 2012, SWEET '12.

[6]  Sriram Krishnamoorthy,et al.  Noncollective Communicator Creation in MPI , 2011, EuroMPI.

[7]  Robert B. Ross,et al.  A configurable algorithm for parallel image-compositing applications , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[8]  Rajeev Thakur,et al.  Efficient Multithreaded Context ID Allocation in MPI , 2012, EuroMPI.

[9]  Robert B. Ross,et al.  A Study of Parallel Particle Tracing for Steady-State and Time-Varying Flow Fields , 2011, 2011 IEEE International Parallel & Distributed Processing Symposium.

[10]  Sriram Krishnamoorthy,et al.  Load Balancing of Dynamical Nucleation Theory Monte Carlo Simulations through Resource Sharing Barriers , 2012, 2012 IEEE 26th International Parallel and Distributed Processing Symposium.

[11]  Ewing Lusk,et al.  More scalability, less pain : A simple programming model and its implementation for extreme computing. , 2010 .

[12]  Robert B. Ross,et al.  Scalable parallel building blocks for custom data analysis , 2011, 2011 IEEE Symposium on Large Data Analysis and Visualization.

[13]  Gregor von Laszewski,et al.  Swift: Fast, Reliable, Loosely Coupled Parallel Computation , 2007, 2007 IEEE Congress on Services (Services 2007).

[14]  Justin M. Wozniak,et al.  Coasters: Uniform Resource Provisioning and Access for Clouds and Grids , 2011, 2011 Fourth IEEE International Conference on Utility and Cloud Computing.

[15]  Ian T. Foster,et al.  Compiler optimization for distributed dynamic data flow programs , 2013 .

[16]  Karsten Schwan,et al.  Adaptable, metadata rich IO methods for portable high performance IO , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[17]  Daniel S. Katz,et al.  Swift: A language for distributed parallel scripting , 2011, Parallel Comput..

[18]  Prasanna Balaprakash,et al.  SPAPT: Search Problems in Automatic Performance Tuning , 2012, ICCS.