Compiling for distributed-memory systems

Compilation techniques for the source-to-source translation of programs in an extended FORTRAN 77 to equivalent parallel message-passing programs are discussed. A machine-independent language extension to FORTRAN 77, Data Parallel FORTRAN (DPF), is introduced. It allows the user to write programs for distributed-memory multiprocessing systems (DMMPS) using global addresses, and to specify the distribution of data across the processors of the machine. Message-Passing FORTRAN (MPF), a FORTRAN extension that allows the formulation of explicitly parallel programs that communicate via explicit message passing, is also introduced. Procedures and optimization techniques for both languages are discussed. Additional optimization methods and advanced parallelization techniques, including run-time analysis, are also addressed. An extensive overview of related work is given. >

[1]  John H. Merlin,et al.  ADAPTing Fortran 90 Array Programs for Distributed Memory Architectures , 1991, ACPC.

[2]  Wolfgang K. Giloi.  SUPRENUM: A trendsetter in modern supercomputer development , 1988, Parallel Comput..

[3]  Michael Gerndt,et al.  Array distribution in SUPERB , 1989, ICS '89.

[4]  Pradip Bose,et al.  Interactive program improvement via EAVE: an expert adviser for vectorization , 1988, ICS '88.

[5]  Thomas Rauber,et al.  Automatic Parallelization for Distributed Memory Multiprocessors , 1994, Automatic Parallelization.

[6]  Thomas G. Macdonald,et al.  MPP Fortran Programming Model , 1992 .

[7]  Michael Gerndt,et al.  Work distribution in parallel programs for distributed memory multiprocessors , 1991, ICS '91.

[8]  Dennis Gannon,et al.  Applying AI Techniques to Program Optimization for Parallel Computers , 1987 .

[9]  Alfred V. Aho,et al.  Compilers: Principles, Techniques, and Tools , 1986, Addison-Wesley series in computer science / World student series edition.

[10]  B. Greer,et al.  Apply: A Parallel Compiler on iWarp for Image-Processing Applications , 1991, The Sixth Distributed Memory Computing Conference, 1991. Proceedings.

[11]  Harry Berryman,et al.  Distributed Memory Compiler Design for Sparse Problems , 1995, IEEE Trans. Computers.

[12]  Guy L. Steele,et al.  Data Optimization: Allocation of Arrays to Reduce Communication on SIMD Machines , 1990, J. Parallel Distributed Comput..

[13]  Jean-Louis Pazat,et al.  Pandore: a system to manage data distribution , 1990, ICS '90.

[14]  Pradip Bose,et al.  Heuristic Rule-Based Program Transformations for Enhanced Vectorization , 1988, International Conference on Parallel Processing.

[15]  Elias N. Houstis,et al.  DOMAIN DECOMPOSER: A Software Tool for Mapping PDE Computations to Parallel Architectures , 1990 .

[16]  John R. Rice,et al.  Parallel ELLPACK: A Develoment and Problem Solving Environment for High Performance Computing Machines , 1991, Programming Environments for High-Level Scientific Problem Solving.

[17]  Allen D. Malony,et al.  Faust: an integrated environment for parallel programming , 1989, IEEE Software.

[18]  Hans P. Zima,et al.  Automatic Support for Data Distribution , 1991, The Sixth Distributed Memory Computing Conference, 1991. Proceedings.

[19]  Charles Koelbel,et al.  Supporting shared data structures on distributed memory architectures , 1990, PPOPP '90.

[20]  Yukio Umetani,et al.  Paragram: A high-level programming language for parallel processors , 1989, Systems and Computers in Japan.

[21]  Santosh G. Abraham,et al.  Compiler techniques for data partitioning of sequentially iterated parallel loops , 1990, ICS '90.

[22]  Barbara M. Chapman,et al.  Programming in Vienna Fortran , 1992, Sci. Program..

[23]  Bobby Schnabel,et al.  An Overview of Dino - A New Language for Numerical Computation on Distributed Memory Multiprocessors , 1987, PPSC.

[24]  Prithviraj Banerjee,et al.  Automatic Data Partitioning on Distributed Memory Multiprocessors , 1991, The Sixth Distributed Memory Computing Conference, 1991. Proceedings.

[25]  Barbara M. Chapman,et al.  Supercompilers for parallel and vector computers , 1990, ACM Press frontier series.

[26]  M. Gerndt,et al.  SUPERB support for irregular scientific computations , 1992, Proceedings Scalable High Performance Computing Conference SHPCC-92..

[27]  M. Baber.  Hypertasking Support for Dynamically Redistributable and Resizeable Arrays on the iPSC , 1991, The Sixth Distributed Memory Computing Conference, 1991. Proceedings.

[28]  Anne Rogers,et al.  Process decomposition through locality of reference , 1989, PLDI '89.

[29]  Guido Wirtz,et al.  Automatic transformation of high-level object-oriented specifications into parallel programs , 1989, Parallel Comput..

[30]  Terrence W. Pratt.  The Pisces 2 Parallel Programming Environment , 1987, ICPP.

[31]  Michael Gerndt,et al.  Updating Distributed Variables in Local Computations , 1990, Concurr. Pract. Exp..

[32]  Arjan J. C. van Gemund,et al.  A method for parallel program generation with an application to the Booster language , 1990, ICS '90.

[33]  J. Ramanujam,et al.  Compile-Time Techniques for Data Distribution in Distributed Memory Machines , 1991, IEEE Trans. Parallel Distributed Syst..

[34]  Geoffrey C. Fox,et al.  An Automatic and Symbolic Parallelization System for Distributed Memory Parallel Computers , 1990, Proceedings of the Fifth Distributed Memory Computing Conference, 1990..

[35]  Jingke Li.  Compiling crystal for distributed-memory machines , 1992 .

[36]  Philip J. Hatcher,et al.  Data-Parallel Programming on MIMD Computers , 1991, IEEE Trans. Parallel Distributed Syst..

[37]  Marina C. Chen.  Optimizing FORTRAN-90 Programs for Data Motion on Massively Parallel Systems , 1992 .

[38]  Charles Koelbel,et al.  Compiling Global Name-Space Parallel Loops for Distributed Execution , 1991, IEEE Trans. Parallel Distributed Syst..

[39]  Michael Gerndt,et al.  Optimizing Communication in Superb , 1990, CONPAR.

[40]  Nicholas Carriero,et al.  Linda and Friends , 1986, Computer.

[41]  H. W. Kuhn.  Solvability and Consistency for Linear Equations and Inequalities , 1956 .

[42]  Piyush Mehrotra,et al.  Vienna Fortran—a Fortran language extension for distributed memory multiprocessors , 1992 .

[43]  David A. Padua,et al.  Advanced compiler optimizations for supercomputers , 1986, CACM.

[44]  Harry Berryman,et al.  Multiprocessors and run-time compilation , 1991, Concurr. Pract. Exp..

[45]  Marco Annaratone,et al.  Parallelization of FORTRAN code on distributed-memory parallel processors , 1990, ICS '90.

[46]  Barbara M. Chapman,et al.  Software Tools for Parallel Program Development , 1991, Programming Environments for High-Level Scientific Problem Solving.

[47]  Michael Gerndt,et al.  SUPERB: A tool for semi-automatic MIMD/SIMD parallelization , 1988, Parallel Comput..

[48]  Marina C. Chen,et al.  Compiling Communication-Efficient Programs for Massively Parallel Machines , 1991, IEEE Trans. Parallel Distributed Syst..

[49]  Jingke Li,et al.  Index domain alignment: minimizing cost of cross-referencing between distributed arrays , 1990, [1990 Proceedings] The Third Symposium on the Frontiers of Massively Parallel Computation.

[50]  Piyush Mehrotra,et al.  Programming distributed memory architectures using Kali , 1990 .

[51]  M. Gerndt.  Program analysis and transformations for message-passing programs , 1992, Proceedings Scalable High Performance Computing Conference SHPCC-92..

[52]  Siegfried Benkner,et al.  Vienna Fortran 90 , 1992, Proceedings Scalable High Performance Computing Conference SHPCC-92..

[53]  P. S. Tseng.  A systolic array programming language , 1990, [1990] Proceedings of the International Conference on Application Specific Array Processors.

[54]  SaltzJoel,et al.  Run-time scheduling and execution of loops on message passing machines , 1990 .

[55]  Ko-Yang Wang,et al.  A Framework for Intelligent Parallel Compilers , 1990 .

[56]  E. DeBenedictis,et al.  nCUBE's Parallel I/O with Unix Compatibility , 1991, The Sixth Distributed Memory Computing Conference, 1991. Proceedings.

[57]  Ken Kennedy,et al.  Compiling Fortran D for MIMD distributed-memory machines , 1992, CACM.

[58]  R. Duffin.  On fourier’s analysis of linear inequality systems , 1974 .