On the use of diagnostic dependence-analysis tools in parallel programming: Experiences using PTOOL

Although considerable technology has been developed for debugging and developing sequential programs, producing verifiably correct parallel code is a much harder task. In view of the large number of possible scheduling sequences, exhaustive testing is not a feasible method for determining whether a given parallel program is correct; nor have there been sufficient theoretical developments to allow the automatic verification of parallel programs. PTOOL, a tool being developed at Rice University in collaboration with users at Los Alamos National Laboratory, provides an alternative mechanism for producing correct parallel code. PTOOL is a semi-automatic tool for detecting implicit parallelism in sequential Fortran code. It uses vectorizing compiler techniques to identify dependences preventing the parallelization of sequential regions. According to the model supported by PTOOL, a programmer should first implement and test his program using traditional sequential debugging techniques. Then, using PTOOL, he can select loop bodies that can be safely executed in parallel. At Los Alamos, we have been interested in examining the role of dependence-analysis tools in the parallel programming process. Therefore, we have used PTOOL as a static debugging tool to analyze parallel Fortran programs. Our experiences using PTOOL lead us to conclude that dependence-analysis tools are useful to today's parallel programmers. Dependence-analysis is particularly useful in the development of asynchronous parallel code. With a tool like PTOOL, a programmer can guarantee that processor scheduling cannot affect the results of his parallel program. If a programmer wishes to implement a partially parallelized region through the use of synchronization primitives, however, he will find that dependence analysis is less useful. While a dependence-analysis tool can greatly simplify the task of writing synchronization code, the ultimate responsibility of correctness is left to the programmer.

[1]  John Randal Allen,et al.  Dependence analysis for subscripted variables and its application to program transformations , 1983 .

[2]  Arthur J. Bernstein,et al.  Analysis of Programs for Parallel Processing , 1966, IEEE Trans. Electron. Comput..

[3]  Ken Kennedy,et al.  PFC: A Program to Convert Fortran to Parallel Form , 1982 .

[4]  Ii R. G. Babb.  Programming the HEP with large-grain data flow techniques , 1985 .

[5]  Ken Kennedy,et al.  Performance of parallel processors , 1989, Parallel Comput..

[6]  Hendrik Pieter Barendregt,et al.  Functional Programming and the Language TALE , 1986, Current Trends in Concurrency.

[7]  E. L. Lusk,et al.  Use of monitors in FORTRAN: a tutorial on the barrier, self-scheduling DO-loop, and askfor monitors , 1985 .

[8]  David A. Padua,et al.  Dependence graphs and compiler optimizations , 1981, POPL '81.

[9]  Raymond L. Murray,et al.  Particle-transport simulation with the Monte Carlo method By and . Technical Information Center, ERDA, 1975, (TID-26607) 115 pp. $dollar;5.45 , 1977 .

[10]  Ronald Gary Cytron.  Compile-time scheduling and optimization for asynchronous machines (multiprocessor, compiler, parallel processing) , 1984 .

[11]  James W. Moore,et al.  Los Alamos experiences with the HEP computer , 1985 .

[12]  Arvind,et al.  Future Scientific Programming on Parallel Machines , 1988, J. Parallel Distributed Comput..

[13]  B. R. Wienke,et al.  Parallel S n iteration schemes , 1990 .

[14]  Ken Kennedy,et al.  PTOOL : A Semi-Automatic Parallel Programming Assistant , 1986, International Conference on Parallel Processing.

[15]  B. R. Wienke,et al.  Parallel S /sub n/ iteration schemes , 1985 .

[16]  David A. Mandell.  Experiences and Results Multitasking a Hydrodynamics Code on Global and Local Memory Machines , 1987, ICPP.

[17]  Daniel P. Friedman,et al.  Aspects of Applicative Programming for Parallel Processing , 1978, IEEE Transactions on Computers.

[18]  David L. Kuck,et al.  The Structure of Computers and Computations , 1978 .

[19]  Ken Kennedy,et al.  Automatic translation of FORTRAN programs to vector form , 1987, TOPL.

[20]  Harry F. Jordan,et al.  Structuring parallel algorithms in an MIMD, shared memory environment , 1986, Parallel Comput..

[21]  Paul O. Frederickson,et al.  A parallel Monte Carlo transport algorithm using a pseudo-random tree to guarantee reproducibility , 1987, Parallel Comput..

[22]  Janusz S. Kowalik.  A Brief Survey of Implicit Parallelism Detection , 1985 .