Compiler Optimization of Dynamic Data Distributions for Distributed-Memory Multicomputers

For distributed-memory multicomputers, the quality of the data partitioning for a given application is crucial to obtaining high performance. This task has traditionally been the user's responsibility, but in recent years much effort has been directed to automating the selection of data partitioning schemes. Several researchers have proposed systems that are able to produce data distributions that remain in effect for the entire execution of an application. For complex programs, however, such static data distributions may be insufficient to obtain acceptable performance. The selection of distributions that dynamically change over the course of a program's execution adds another dimension to the data partitioning problem. In this chapter we present an approach for selecting dynamic data distributions as well as a technique for analyzing the resulting data redistribution in order to generate efficient code.

[1]  Ken Kennedy,et al.  Compiling Fortran D for MIMD distributed-memory machines , 1992, CACM.

[2]  Mark N. Wegman,et al.  Efficiently computing static single assignment form and the control dependence graph , 1991, TOPL.

[3]  Ken Kennedy,et al.  Evaluation of compiler optimizations for Fortran D on MIMD distributed memory machines , 1992, ICS '92.

[4]  Chau-Wen Tseng.  An optimizing Fortran D compiler for MIMD distributed-memory machines , 1993 .

[5]  Eduard Ayguadé,et al.  Data Redistribution in an Automatic Data Distribution Tool , 1995, LCPC.

[6]  Guy L. Steele,et al.  Data Optimization: Allocation of Arrays to Reduce Communication on SIMD Machines , 1990, J. Parallel Distributed Comput..

[7]  E. Ayguade,et al.  A Novel Approach Towards Automatic Data Distribution , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[8]  Ping-Sheng Tseng.  Compiling programs for a linear systolic array , 1990, PLDI '90.

[9]  John A. Chandy,et al.  Communication Optimizations Used in the Paradigm Compiler for Distributed-Memory Multicomputers , 1994, 1994 Internatonal Conference on Parallel Processing Vol. 2.

[10]  John A. Chandy,et al.  The Paradigm Compiler for Distributed-Memory Multicomputers , 1995, Computer.

[11]  KennedyKen,et al.  GIVE-N-TAKEa balanced code placement framework , 1994 .

[12]  Milind Girkar,et al.  Parafrase-2: an Environment for Parallelizing, Partitioning, Synchronizing, and Scheduling Programs on Multiprocessors , 1989, Int. J. High Speed Comput..

[13]  Joseph A. Fisher,et al.  Trace Scheduling: A Technique for Global Microcode Compaction , 1981, IEEE Transactions on Computers.

[14]  R. Sadourny.  The Dynamics of Finite-Difference Models of the Shallow-Water Equations , 1975 .

[15]  Ken Kennedy,et al.  Automatic Data Layout Using 0-1 Integer Programming , 1994, IFIP PACT.

[16]  Santosh G. Abraham,et al.  Compiling Parallel Loops for High Performance Computers , 1993 .

[17]  Barbara M. Chapman,et al.  Automatic Support for Data Distribution on Distributed Memory Multiprocessor Systems , 1993, LCPC.

[18]  Robert E. Tarjan,et al.  Data structures and network algorithms , 1983, CBMS-NSF regional conference series in applied mathematics.

[19]  John R. Gilbert,et al.  Automatic array alignment in data-parallel programs , 1993, POPL '93.

[20]  Guy L. Steele,et al.  The High Performance Fortran Handbook , 1993 .

[21]  Prithviraj Banerjee,et al.  Interprocedural Array Redistribution Data-Flow Analysis , 1996, LCPC.

[22]  TsengPing-Sheng.  Compiling programs for a linear systolic array , 1990 .

[23]  Balachander Krishnamurthy,et al.  Practical reusable UNIX software , 1995 .

[24]  Skef Wholey.  Automatic data mapping for distributed-memory parallel computers , 1992, ICS '92.

[25]  Manish Gupta,et al.  Compile-time estimation of communication costs on multicomputers , 1992, Proceedings Sixth International Parallel Processing Symposium.

[26]  Monica S. Lam,et al.  Global optimizations for parallelism and locality on scalable parallel machines , 1993, PLDI '93.

[27]  Alfred V. Aho,et al.  Compilers: Principles, Techniques, and Tools , 1986, Addison-Wesley series in computer science / World student series edition.

[28]  Ken Kennedy,et al.  Interprocedural compilation of Fortran D for MIMD distributed-memory machines , 1992, Proceedings Supercomputing '92.

[29]  Gene H. Golub,et al.  Scientific computing: an introduction with parallel computing , 1993 .

[30]  Prithviraj Banerjee,et al.  Dynamic Data Partitioning for Distributed-Memory Multicomputers , 1996, J. Parallel Distributed Comput..

[31]  Keshav Pingali,et al.  Solving Alignment Using Elementary Linear Algebra , 2001, Compiler Optimizations for Scalable Parallel Systems Languages.

[32]  Prithviraj Banerjee,et al.  Compiler techniques for optimizing communication and data distribution for distributed-memory multicomputers , 1996 .

[33]  Manish Gupta,et al.  PARADIGM: a compiler for automatic data distribution on multicomputers , 1993, ICS '93.

[34]  J. Ramanujam,et al.  Compile-Time Techniques for Data Distribution in Distributed Memory Machines , 1991, IEEE Trans. Parallel Distributed Syst..

[35]  K. Kennedy,et al.  Automatic Data Layout for High Performance Fortran , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[36]  Eugene Wilson Hodges,et al.  High Performance Fortran Support for the PARADIGM Compiler , 1995 .

[37]  Santosh G. Abraham,et al.  Compiling Parallel Loops for High Performance Computers: Partitioning, Data Assignment and Remapping , 1992 .

[38]  Prithviraj Banerjee,et al.  Automatic generation of efficient array redistribution routines for distributed memory multicomputers , 1995, Proceedings Frontiers '95. The Fifth Symposium on the Frontiers of Massively Parallel Computation.

[39]  Manish Gupta,et al.  Automatic Data Partitioning on Distributed Memory Multicomputers , 1992 .

[40]  Ken Kennedy,et al.  GIVE-N-TAKE—a balanced code placement framework , 1994, PLDI '94.

[41]  Marina C. Chen,et al.  The Data Alignment Phase in Compiling Programs for Distrubuted-Memory Machines , 1991, J. Parallel Distributed Comput..