An expression-rewriting framework to generate communication sets for HPF programs with block-cyclic distribution

In this paper, we present a new framework based on expression rewritings and a calculus form called CSD calculus to generate the local enumeration set and communication set for HPF programs with Block-Cyclic distribution. Our framework is a practical software framework, and can handle the general cases so that the communication set of HPF programs of "Block-Cyclic" distributions with two-level alignments (or multiple-level alignments), multidimensional arrays, array intrinsic functions (such as Transpose operation), and affine indexes and axis exchange in the array subscript, can be calculated in a systematic way with a sound software foundation. Previously, existing work do not report a software framework to solve a problem with such general cases. In addition, our expression-rewriting framework is based on a new representative form, CSD (common-stride descriptor), to describe the regularity of the access patterns of HPF programs with "Block-Cyclic" distribution. We also demonstrate a calculus of CSD that CSD is closed under intersection and normalization, which helps the process of calculating local enumeration and communication sets of HPF programs with Block-Cyclic distributions. Experimental results show that our software scheme not only can be easily implemented in the practice, but also is with good efficiency.

[1]  Jenq Kuen Lee,et al.  Data distribution analysis and optimization for Pointer-based distributed programs , 1997, Proceedings of the 1997 International Conference on Parallel Processing (Cat. No.97TB100162).

[2]  Sandeep K. S. Gupta,et al.  On Compiling Array Expressions for Efficient Execution on Distributed-Memory Machines , 1993, 1993 International Conference on Parallel Processing - ICPP'93.

[3]  Jerrold L. Wagener,et al.  Fortran 90 Handbook: Complete Ansi/Iso Reference , 1992 .

[4]  Ken Kennedy,et al.  Compiling Fortran D for MIMD distributed-memory machines , 1992, CACM.

[5]  Marina C. Chen,et al.  The Data Alignment Phase in Compiling Programs for Distrubuted-Memory Machines , 1991, J. Parallel Distributed Comput..

[6]  Jenq Kuen Lee,et al.  Object oriented parallel programming: experiments and results , 1991, Proceedings of the 1991 ACM/IEEE Conference on Supercomputing (Supercomputing '91).

[7]  Ken Kennedy,et al.  A linear-time algorithm for computing the memory access sequence in data-parallel programs , 1995, PPOPP '95.

[8]  Barbara M. Chapman,et al.  Supercompilers for parallel and vector computers , 1990, ACM Press frontier series.

[9]  Jang-Ping Sheu,et al.  Table-Lookup Approach for Compiling Two-Level Data-Processor Mappings in HPF , 1997, LCPC.

[10]  Jenq Kuen Lee,et al.  Integrating Automatic Data Alignment and Array Operation Synthesis to Optimize Data Parallel Programs , 1997, LCPC.

[11]  R. van de Geijn,et al.  A look at scalable dense linear algebra libraries , 1992, Proceedings Scalable High Performance Computing Conference SHPCC-92..

[12]  J. Ramanujam,et al.  Fast Address Sequence Generation for Data-Parallel Programs Using Integer Lattices , 1995, LCPC.

[13]  Dennis Gannon,et al.  Distributed pC++ Basic Ideas for an Object Parallel Language , 1993, Sci. Program..

[14]  Samuel P. Midkiff.  Local Iteration Set Computation for Block-Cyclic Distributions , 1995, ICPP.

[15]  Jenq Kuen Lee,et al.  An array operation synthesis scheme to optimize Fortran 90 programs , 1995, PPOPP '95.

[16]  Ken Kennedy,et al.  Compilation techniques for block-cyclic distributions , 1994 .

[17]  Jenq Kuen Lee,et al.  Array operation synthesis to optimize HPF programs , 1996, Proceedings of the 1996 ICPP Workshop on Challenges for Parallel Processing.

[18]  Lei Wang,et al.  Runtime Performance of Parallel Array Assignment: An Empirical Study , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[19]  Henk J. Sips,et al.  Analysis of Local Enumeration and Storage Schemes in HPF , 1998, Parallel Comput..

[20]  Jenq Kuen Lee,et al.  Parallel Array Object I/O Support on Distributed Environments , 1997, J. Parallel Distributed Comput..

[21]  Geoffrey C. Fox,et al.  Compiling Fortran 90D/HPF for Distributed Memory MIMD Computers , 1994, J. Parallel Distributed Comput..

[22]  Ken Kennedy,et al.  Communication Generation for Cyclic(K) Distributions , 1996 .

[23]  Sandeep K. S. Gupta,et al.  On Compiling Array Expressions for Efficient Execution on Distributed-Memory Machines , 1993, 1993 International Conference on Parallel Processing - ICPP'93.

[24]  Guy L. Steele,et al.  The High Performance Fortran Handbook , 1993 .

[25]  Thomas R. Gross,et al.  Generating Communication for Array Statement: Design, Implementation, and Evaluation , 1994, J. Parallel Distributed Comput..

[26]  John R. Gilbert,et al.  Generating local addresses and communication sets for data-parallel programs , 1993, PPOPP '93.

[27]  Charles Koelbel.  Compile-time generation of regular communications patterns , 1991, Proceedings of the 1991 ACM/IEEE Conference on Supercomputing (Supercomputing '91).

[28]  Jenq Kuen Lee,et al.  Sampling and Analytical Techniques for Data Distribution of Parallel Sparse Computation , 1997, PPSC.

[29]  Henk J. Sips,et al.  An Implementation Framework for HPF Distributed Arrays on Message-Passing Parallel Computer Systems , 1996, IEEE Trans. Parallel Distributed Syst..