A template for non-uniform parallel loops based on dynamic scheduling and prefetching techniques

In this paper we present an efficient template for the implement ation on distribut cd-memory multiprocessors of n on– uniform parallei loops, i.e. loops whose independent iterations are characterized by highly varying execution times. The template relies upon a static blocking distribution of array data sets to exploit locality, and a hybrid scheduling policy to smooth uneven processor finishing time. It initially adopts a static scheduling technique based on the owner computes rzde. As soon as a workload imbalance is detected, it exploits a dynamic receiver wzitiated technique to move work towards unloaded processors. Prefetching is used to reduce overheads due to the communications needed to monitor the load, move iterations, and restore the consistency of migrated data. Accurate performance costs of the technique can be derived, thus allowing the template to be used by a compiler to generate well-balanced code for non–uniform parallel loops. Experiments were conducted on a 64-node Cray T3D, and the performance of the proposed template was compared with the one obtained by using the CRAFT-Fortran language (an HPF-like language).