Strategies for retargeting of existing sequential programs for parallel processing

There are relative advantages and disadvantages of small-grain and large-grain parallelism. It is well established that, for MIMD machines, small-grain parallelism is not recommended because of associated excessive interprocessor communication overhead. On the other hand, the large-grain approach does not provide an adequate degree of parallelism and may not provide necessary speedup. In our work, we have adopted an optimal-grain approach such that the parallelism obtained at small-grain level is retained while minimizing the communication overhead. The idea is to employ a systematic approach in partitioning an existing program into a set of large grains such that the best performance in terms of total execution time is achieved by evaluating the tradeoff between parallelism and communication cost. To do this effectively, we introduce a model which can accurately represent the possible communication between various computational units of a program, and can measure possible computational overlap between interacting computational units. The tradeoff between parallelism and communication cost leads to an improved performance. Based on this model, software packages have been developed to accept a program written in FORTRAN, to analyze its data dependency, and to partition the program into a set of large grains. Extensive experiments conducted on EISPACK subroutines show substantial improvement in execution time on MIMD machines.