Software support for parallel processing of irregular and dynamic computations

Many real world scientific computations are irregular and dynamic, which pose great challenge to the effort of parallelization. In this thesis we study the efficient mapping of a subclass of these problems, namely the "stepwise slowly changing" problems, onto distributed memory multiprocessors using the task graph scheduling approach. There exists a large class of applications which belong to this category. Intuitively, the irregularity requires sophisticated mapping algorithms, and the "slowness" in the changes of the computational structures between steps allows the scheduling cost to be amortized, justifying the approach. We study three representative and widely-used applications: The N-body simulation in astrophysics, the Vortex-Sheet Roll-Up and the Contour Dynamics Computation from Computational Fluid Dynamics. We start with an initial global compile-time scheduling, and apply new rescheduling algorithms to improve performance when this schedule degenerates over the iterative process. We develop rescheduling algorithms for two important dynamic patterns: task graph weight variation, and dynamic spawning of new subgraphs. These algorithms are tested on random graphs and real applications such as the FMM N-body and Vortex Sheet. Our experiments show that global scheduling using sophisticated methods can be beneficial for these problems, and our fast rescheduling algorithms can correct run-time imbalance with very low cost. In summary, we discuss several central issues such as schedule reuse, performance/overhead trade-off and the selection of rescheduling methods. We identify classes of problems where rescheduling algorithms are applicable, and present experimental evidence to justify our approach. Throughout the thesis, performance results are obtained from particular problems but presented in the general framework of software support systems. In addition, we examine an automatic task graph generation tool that can handle restricted cases of sequential code, and carry out its integration with our scheduling system. The new system is capable of realizing automatic parallelization of simple programs, a step forward to the grand challenge of fully automatic parallelization of regular and irregular code.