A Cluster-as-Accelerator Approach for SPMD-Free Data Parallelism

In this paper we present a novel approach for functional-style programming of distributed-memory clusters, targeting data-centric applications. The programming model proposed is purely sequential, SPMD-free and based on high-level functional features introduced since C++11 specification. Additionally, we propose a novel cluster-as-accelerator design principle. In this scheme, cluster nodes act as general interpreters of user-defined functional tasks over node-local portions of distributed data structures. We envision coupling a simple yet powerful programming model with a lightweight, locality-aware distributed runtime as a promising step along the road towards high-performance data analytics, in particular under the perspective of the upcoming exascale era. We implemented the proposed approach in SkeDaTo, a prototyping C++ library of data-parallel skeletons exploiting cluster-as-accelerator at the bottom layer of the runtime software stack.

[1]  Murray Cole,et al.  Algorithmic Skeletons: Structured Management of Parallel Computation , 1989 .

[2]  Message Passing Interface Forum MPI: A message - passing interface standard , 1994 .

[3]  Christoph W. Kessler,et al.  SkePU: a multi-backend skeleton programming library for multi-GPU systems , 2010, HLPP '10.

[4]  Michael J. Franklin,et al.  Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing , 2012, NSDI.

[5]  Marco Danelutto,et al.  An advanced environment supporting structured parallel programming in Java , 2003, Future Gener. Comput. Syst..

[6]  Zhenjiang Hu,et al.  A library of constructive skeletons for sequential style of parallel programming , 2006, InfoScale '06.

[7]  Katherine Yelick,et al.  UPC: Distributed Shared-Memory Programming , 2003 .

[8]  Vivek Sarkar,et al.  X10: an object-oriented approach to non-uniform cluster computing , 2005, OOPSLA '05.

[9]  Patrizio Dazzi,et al.  Scalable Computing: Practice and Experience WSSP, Warsaw, Poland, 2007. To appear. MUSKEL: AN EXPANDABLE SKELETON ENVIRONMENT∗ , 2007 .

[10]  Jarek Nieplocha,et al.  Advances, Applications and Performance of the Global Arrays Shared Memory Programming Toolkit , 2006, Int. J. High Perform. Comput. Appl..

[11]  Bradford L. Chamberlain,et al.  Parallel Programmability and the Chapel Language , 2007, Int. J. High Perform. Comput. Appl..

[12]  Herbert Kuchen,et al.  The Münster Skeleton Library Muesli: A comprehensive overview , 2009 .