Programming Irregular Applications: Runtime Support, Compilation and Tools

Abstract In this chapter, we present a summary of the runtime support, compiler and tools development efforts in the CHAOS group at the University of Maryland. The principal focus of the CHAOS group's research has been to develop tools, compiler runtime support and compilation techniques to help scientists and engineers develop high-speed parallel implementations of codes for irregular scientific problems (i.e. problems that are unstructured, sparse, adaptive or block structured). We have developed a series of runtime support libraries (CHAOS, CHAOS+ +) that carry out the preprocessing and data movement needed to efficiently implement irregular and block structured scientific algorithms on distributed memory machines and networks of workstations. Our compilation research has played a major role in demonstrating that it is possible to develop data parallel compilers able to make effective use of a wide variety of runtime optimizations. We have also been exploring ways to support interoperability between sequential and parallel programs written using different languages and programming paradigms.

[1]  T. von Eicken,et al.  Parallel programming in Split-C , 1993, Supercomputing '93.

[2]  Shahid H. Bokhari,et al.  A Partitioning Strategy for Nonuniform Problems on Multiprocessors , 1987, IEEE Transactions on Computers.

[3]  Joel H. Saltz,et al.  An Integrated Runtime and Compile-Time Approach for Parallelizing Structured and Block Structured Applications , 1995, IEEE Trans. Parallel Distributed Syst..

[4]  Charles Koelbel,et al.  Compiling Global Name-Space Parallel Loops for Distributed Execution , 1991, IEEE Trans. Parallel Distributed Syst..

[5]  Charles Koelbel,et al.  Supporting shared data structures on distributed memory architectures , 1990, PPOPP '90.

[6]  Joel H. Saltz,et al.  Implementation of a parallel unstructured Euler solver on shared- and distributed-memory architectures , 1992, Supercomputing '92.

[7]  Steven J. Plimpton,et al.  Parallel Algorithms for Dynamically Partitioning Unstructured Grids , 1995, PPSC.

[8]  Tao Yang,et al.  Scheduling Of Structured and Unstructured computation , 1994, Interconnection Networks and Mapping and Scheduling Parallel Computations.

[9]  Anne Rogers,et al.  Compiling for Distributed Memory Architectures , 1994, IEEE Trans. Parallel Distributed Syst..

[10]  Barbara M. Chapman,et al.  New data-parallel language features for sparse matrix computations , 1995, Proceedings of 9th International Parallel Processing Symposium.

[11]  Joel H. Saltz,et al.  Run-Time Parallelization and Scheduling of Loops , 1991, IEEE Trans. Computers.

[12]  Gregory V. Wilson,et al.  Parallel Programming Using C , 1996 .

[13]  Katherine A. Yelick,et al.  Portable Parallel Irregular Applications , 1995, PSLS.

[14]  Joel H. Saltz,et al.  Interprocedural partial redundancy elimination and its application to distributed memory compilation , 1995, PLDI '95.

[15]  Jong-Deok Choi,et al.  Global communication analysis and optimization , 1996, PLDI '96.

[16]  Joel H. Saltz,et al.  Runtime Support Dynamic Load Balancing Strategies for Structured Adaptive Applications , 1995, PPSC.

[17]  Ian T. Foster,et al.  Compositional parallel programming languages , 1996, TOPL.

[18]  Jack Dongarra,et al.  MPI: The Complete Reference , 1996 .

[19]  Guy L. Steele,et al.  The High Performance Fortran Handbook , 1993 .

[20]  Scott B. Baden,et al.  Flexible Communication Mechanisms for Dynamic Structured Applications , 1996, IRREGULAR.

[21]  Petter E. Bjørstad,et al.  Proceedings of the Seventh SIAM Conference on Parallel Processing for Scientific Computing, PPSC 1995, San Francisco, California, USA, February 15-17, 1995 , 1995, PPSC.

[22]  Michael Gerndt,et al.  Updating Distributed Variables in Local Computations , 1990, Concurr. Pract. Exp..

[23]  Dennis Gannon,et al.  Distributed pC++ Basic Ideas for an Object Parallel Language , 1993, Sci. Program..

[24]  A. Sussman,et al.  Compiler and runtime support for structured and block structured applications , 1993, Supercomputing '93.

[25]  Aart J. C. Bik,et al.  Automatic Data Structure Selection and Transformation for Sparse Matrix Computations , 1996, IEEE Trans. Parallel Distributed Syst..

[26]  Joel H. Saltz,et al.  Experimental evaluation of efficient sparse matrix distributions , 1996, ICS '96.

[27]  Tao Yang,et al.  PYRROS: static task scheduling and code generation for message passing multiprocessors , 1992 .

[28]  Prithviraj Banerjee,et al.  Exploiting spatial regularity in irregular iterative applications , 1995, Proceedings of 9th International Parallel Processing Symposium.

[29]  Larry S. Davis,et al.  A High Performance Image Database System for Remotely Sensed Imagery , 1996, Euro-Par, Vol. II.

[30]  K. Kennedy,et al.  Index Array Flattening Through Program Transformation , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[31]  Mark Weiser,et al.  Program Slicing , 1981, IEEE Transactions on Software Engineering.

[32]  Ken Kennedy,et al.  Compiler Analysis for Irregular Problems in Fortran D , 1992, LCPC.

[33]  Dimitri J. Mavriplis,et al.  The design and implementation of a parallel unstructured Euler solver using software primitives , 1992 .

[34]  Geoffrey C. Fox,et al.  Supporting irregular distributions using data-parallel languages , 1995, IEEE Parallel Distributed Technol. Syst. Appl..

[35]  Larry S. Davis,et al.  High performance computing for land cover dynamics , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 2 - Conference B: Computer Vision & Image Processing. (Cat. No.94CH3440-5).

[36]  J. Saltz,et al.  Interprocedural Compilation of Irregular Applications for Distributed Memory Machines , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[37]  Geoffrey C. Fox,et al.  Runtime Support and Compilation Methods for User-Specified Irregular Data Distributions , 1995, IEEE Trans. Parallel Distributed Syst..

[38]  Milind Girkar,et al.  Parafrase-2: an Environment for Parallelizing, Partitioning, Synchronizing, and Scheduling Programs on Multiprocessors , 1989, Int. J. High Speed Comput..

[39]  Geoffrey C. Fox,et al.  Compiling Fortran 90D/HPF for Distributed Memory MIMD Computers , 1994, J. Parallel Distributed Comput..

[40]  Joel H. Saltz,et al.  Runtime and language support for compiling adaptive irregular programs on distributed‐memory machines , 1995, Softw. Pract. Exp..

[41]  Harry Berryman,et al.  Distributed Memory Compiler Design for Sparse Problems , 1995, IEEE Trans. Computers.

[42]  Joel H. Saltz,et al.  Parallelizing Molecular Dynamics Programs for Distributed Memory Machines: An Application of the Cha , 1994 .

[43]  Joel H. Saltz,et al.  Efficient runtime support for parallelizing block structured applications , 1994, Proceedings of IEEE Scalable High Performance Computing Conference.

[44]  Joel H. Saltz,et al.  Principles of runtime support for parallel processors , 1988, ICS '88.

[45]  R. Parsons,et al.  A++/P++ array classes for architecture independent finite difference computations , 1994 .

[46]  Katherine Yelick,et al.  Multipol: A Distributed Data Structure Library , 1995 .

[47]  Ken Kennedy,et al.  Computer support for machine-independent parallel programming in Fortran D , 1992 .

[48]  H. D. Simon,et al.  A spectral algorithm for envelope reduction of sparse matrices , 1993, Supercomputing '93. Proceedings.

[49]  Michael S. Woronowicz,et al.  Spacecraft contamination investigation by direct simulation Monte Carlo - Contamination on UARS/HALOE , 1993 .

[50]  Scott B. Baden,et al.  The Parallelization of an Adaptive Multigrid Eigenvalue Solver with LPARX , 1995, PPSC.

[51]  Joel H. Saltz,et al.  Runtime coupling of data-parallel programs , 1995, ICS '96.

[52]  Joel H. Saltz,et al.  Run-Time Techniques for Parallelizing Sparse Matrix Problems , 1995, IRREGULAR.

[53]  M. Gerndt,et al.  SUPERB support for irregular scientific computations , 1992, Proceedings Scalable High Performance Computing Conference SHPCC-92..

[54]  Joel H. Saltz,et al.  Interprocedural Communication Optimizations for Distributed Memory Compilation , 1994, LCPC.

[55]  Joel H. Saltz,et al.  Communication Optimizations for Irregular Scientific Computations on Distributed Memory Architectures , 1994, J. Parallel Distributed Comput..

[56]  Prithviraj Banerjee,et al.  Compiler support for hybrid irregular accesses on multicomputers , 1996, ICS '96.

[57]  Joel H. Saltz,et al.  Interoperability of data parallel runtime libraries with meta-chaos , 1996 .

[58]  Lie Wang,et al.  Towards a fast implementation of spectral nested dissection , 1992, Proceedings Supercomputing '92.