Tlib - a library to support programming with hierarchical multi-processor tasks

The paper considers the modular programming with hierarchically structured multi-processor tasks on top of SPMD tasks for distributed memory machines. The parallel execution requires a corresponding decomposition of the set of processors into a hierarchical group structure onto which the tasks are mapped. The result is a multi-level group SPMD computation model with varying processor group structures. The advantage of this kind of mixed task and data parallelism is a potential to reduce the communication overhead and to increase scalability. We present a runtime library to support the coordination of hierarchically structured multi-processor tasks. The library exploits an extended parallel group SPMD programming model and manages the entire task execution including the dynamic hierarchy of processor groups. The library is built on top of MPI, has an easy-to-use interface, and leads to only a marginal overhead while allowing static planning and dynamic restructuring.

[1]  Scott B. Baden,et al.  A Programming Methodology for Dual-Tier Multicomputers , 2000, IEEE Trans. Software Eng..

[2]  William F. McColl.  Universal Computing , 1996, Euro-Par, Vol. I.

[3]  David B. Skillicorn,et al.  Questions and Answers about BSP , 1997, Sci. Program..

[4]  Thomas Rauber,et al.  Performance modelling for task-parallel programs , 2004 .

[5]  Constantine D. Polychronopoulos,et al.  Microarchitecture support for dynamic scheduling of acyclic task graphs , 1992, MICRO.

[6]  Beverly Clayton.  Proceedings of the 1996 ACM/IEEE Conference on Supercomputing, November 17-22, 1996, Pittsburgh, PA, USA , 1996, SC.

[7]  Christoph W. Kessler,et al.  NestStep: Nested Parallelism and Virtual Shared Memory for the BSP Model , 2000, The Journal of Supercomputing.

[8]  Jaspal Subhlok,et al.  A new model for integrated nested task and data parallel programming , 1997, PPOPP '97.

[9]  James Demmel,et al.  Modeling the benefits of mixed data and task parallelism , 1995, SPAA '95.

[10]  Zhiwei Xu,et al.  Early Prediction of MPP Performance: Th SP2, T3D, and Paragon Experiences , 1996, Parallel Comput..

[11]  V. Strassen.  Gaussian elimination is not optimal , 1969 .

[12]  Ben P. Sommeijer,et al.  Embedded diagonally implicit Runge-Kutta algorithms on parallel computers , 1989 .

[13]  Thomas Rauber,et al.  ORT: a communication library for orthogonal processor groups , 2001, SC '01.

[14]  Frederica Darema,et al.  A single-program-multiple-data computational model for EPEX/FORTRAN , 1988, Parallel Comput..

[15]  Ian Foster,et al.  A compilation system that integrates High Performance Fortran and Fortran M , 1994, Proceedings of IEEE Scalable High Performance Computing Conference.

[16]  Prithviraj Banerjee,et al.  Simultaneous exploitation of task and data parallelism in regular scientific applications , 1996 .

[17]  Scott B. Baden,et al.  A programming model for block-structured scientific calculations on smp clusters , 1998 .

[18]  Matthew Haines,et al.  Approaches for integrating task and data parallelism , 1998, IEEE Concurr..

[19]  Alok N. Choudhary,et al.  Double Standards: Bringing Task Parallelism to HPF Via the Message Passing Interface , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[20]  Jaspal Subhlok.  Automatic Mapping of Task and Data Parallel Programs for Efficient Execution on Multicomputers , 1993 .

[21]  E. Hairer,et al.  Solving ordinary differential equations I (2nd revised. ed.): nonstiff problems , 1993 .

[22]  Thomas Rauber,et al.  Diagonal-Implicitly Iterated Runge-Kutta Methods on Distributed Memory Machines , 1999, Int. J. High Speed Comput..

[23]  Geoffrey C. Fox,et al.  A High Level SPMD Programming Model: HPspmd and its Java Language Binding , 1998 .

[24]  Thomas Rauber,et al.  Load balancing schemes for extrapolation methods , 1997, Concurr. Pract. Exp..

[25]  Prithviraj Banerjee,et al.  PARADIGM (version 2.0): a new HPF compilation system , 1999, Proceedings 13th International Parallel Processing Symposium and 10th Symposium on Parallel and Distributed Processing. IPPS/SPDP 1999.

[26]  Scott B. Baden,et al.  Irregular Coarse-Grain Data Parallelism under LPARX , 1996, Sci. Program..

[27]  Ernst Hairer,et al.  Solving Ordinary Differential Equations I: Nonstiff Problems , 2009 .

[28]  K. Mani Chandy,et al.  Fortran M: A Language for Modular Parallel Programming , 1995, J. Parallel Distributed Comput..

[29]  Lola Lemire Tostevin.  Double standards , 1985 .

[30]  Scott B. Baden,et al.  Multiple data parallelism with HPF and KeLP , 1999, Future Gener. Comput. Syst..

[31]  David B. Skillicorn,et al.  Models and languages for parallel computation , 1998, CSUR.