ADDT: automatic data distribution tool for porting programs to PVM

An important issue facing users who develop applications on PVM is that of partitioning the data over the different processors and generating communication statements for non-local data accesses. The burden of generating communication statements is alleviated by data parallel languages like HPF. HPF requires the user to specify the distribution of the data in the application. This distribution is specified over the processors on which the application is to execute. A good data distribution depends on the number of processors, the communication latency and the underlying compilation system. We present a method for automatically generating a dynamic data distribution and alignments for a distributed memory environment. This method has been implemented in an automatic data distribution tool called ADDT. ADDT reads in a shared memory parallel program in Fortran and generates as output a HPF program. Our method works by breaking up the input program into sub parts called distribution blocks (DB). It then obtains a set of data distribution for the arrays in each DB using the data layouts from preceding DB's. The algorithms used to obtain a set of data distribution (called candidate distributions) for a DB are discussed. The final data layout for the program has to be selected from these candidate distributions. This selection is formulated as a linear optimization problem which minimizes the communication while minimizing loss of parallelism. We present experimental results obtained by running ADDT on some example programs.

[1]  Robert J. Harrison,et al.  Global Arrays: a portable "shared-memory" programming model for distributed memory computers , 1994, Proceedings of Supercomputing '94.

[2]  Charles Koelbel,et al.  Compiling Global Name-Space Parallel Loops for Distributed Execution , 1991, IEEE Trans. Parallel Distributed Syst..

[3]  Ken Kennedy,et al.  Parallelization of linearized applications in Fortran D , 1994, Proceedings of 8th International Parallel Processing Symposium.

[4]  Geoffrey C. Fox,et al.  Fortran 90D/HPF compiler for distributed memory MIMD computers: design, implementation, and performance results , 1993, Supercomputing '93.

[5]  Prithviraj Banerjee,et al.  Automatic Selection of Dynamic Data Partitioning Schemes for Distributed-Memory Multicomputers , 1995, LCPC.

[6]  Ken Kennedy,et al.  Interprocedural compilation of Fortran D for MIMD distributed-memory machines , 1992, Proceedings Supercomputing '92.

[7]  Manish Gupta,et al.  PARADIGM: a compiler for automatic data distribution on multicomputers , 1993, ICS '93.

[8]  John R. Gilbert,et al.  Array Distribution in Data-Parallel Programs , 1994, LCPC.

[9]  J. Ramanujam,et al.  Compile-Time Techniques for Data Distribution in Distributed Memory Machines , 1991, IEEE Trans. Parallel Distributed Syst..

[10]  Ken Kennedy,et al.  Automatic Data Layout Using 0-1 Integer Programming , 1994, IFIP PACT.

[11]  Ken Kennedy,et al.  Evaluation of compiler optimizations for Fortran D on MIMD distributed memory machines , 1992, ICS '92.

[12]  Manish Gupta,et al.  Demonstration of Automatic Data Partitioning Techniques for Parallelizing Compilers on Multicomputers , 1992, IEEE Trans. Parallel Distributed Syst..

[13]  Prithviraj Banerjee,et al.  Automatic Data Partitioning on Distributed Memory Multiprocessors , 1991, The Sixth Distributed Memory Computing Conference, 1991. Proceedings.

[14]  Monica S. Lam,et al.  Data and computation transformations for multiprocessors , 1995, PPOPP '95.

[15]  John R. Gilbert,et al.  The Alignment-Distribution Graph , 1993, LCPC.

[16]  John A. Chandy,et al.  Communication Optimizations Used in the Paradigm Compiler for Distributed-Memory Multicomputers , 1994, 1994 Internatonal Conference on Parallel Processing Vol. 2.

[17]  Ken Kennedy,et al.  Compiling Fortran 77D and 90D for MIMD distributed-memory machines , 1992, [Proceedings 1992] The Fourth Symposium on the Frontiers of Massively Parallel Computation.

[18]  Viktor K. Prasanna,et al.  Permutations on Illiac IV-Type Networks , 1986, IEEE Transactions on Computers.

[19]  Ken Kennedy,et al.  Automatic Data Layout for Distributed-Memory Machines in the D Programming Environment , 1994, Automatic Parallelization.

[20]  C. S. RaghavendraSchool.  Addt: Automatic Data Distribution Tool for Porting Programs to Distributed Memory Machines , 1995 .

[21]  E. Ayguade,et al.  A Novel Approach Towards Automatic Data Distribution , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[22]  Ravi Mirchandaney,et al.  Improving the performance of DSM systems via compiler involvement , 1994, Proceedings of Supercomputing '94.

[23]  K. Kennedy,et al.  Automatic Data Layout for High Performance Fortran , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[24]  Paul Feautrier.  Toward automatic partitioning of arrays on distributed memory computers , 1993, ICS '93.

[25]  Marina C. Chen,et al.  The Data Alignment Phase in Compiling Programs for Distrubuted-Memory Machines , 1991, J. Parallel Distributed Comput..

[26]  Ulrich Kremer,et al.  NP-completeness of Dynamic Remapping , 1993 .

[27]  Michael Gerndt,et al.  SUPERB: A tool for semi-automatic MIMD/SIMD parallelization , 1988, Parallel Comput..