Task Clustering and Scheduling for Distributed Memory Parallel Architectures

This paper addresses the problem of scheduling parallel programs represented as directed acyclic task graphs for execution on distributed memory parallel architectures. Because of the high communication overhead in existing parallel machines, a crucial step in scheduling is task clustering, the process of coalescing fine grain tasks into single coarser ones so that the overall execution time is minimized. The task clustering problem is NP-hard, even when the number of processors is unbounded and task duplication is allowed. A simple greedy algorithm is presented for this problem which, for a task graph with arbitrary granularity, produces a schedule whose makespan is at most twice optimal. Indeed, the quality of the schedule improves as the granularity of the task graph becomes larger. For example, if the granularity is at least 1/2, the makespan of the schedule is at most 5/3 times optimal. For a task graph with n tasks and e inter-task communication constraints, the algorithm runs in O(n(n lg n+e)) time, which is n times faster than the currently best known algorithm for this problem. Similar algorithms are developed that produce: (1) optimal schedules for coarse grain graphs; (2) 2-optimal schedules for trees with no task duplication; and (3) optimal schedules for coarse grain trees with no task duplication.

[1]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[2]  Vivek Sarkar,et al.  Partitioning and scheduling parallel programs for execution on multiprocessors , 1987 .

[3]  Jing-Jang Hwang,et al.  Multiprocessor scheduling with interprocessor communication delays , 1988 .

[4]  Boontee Kruatrachue,et al.  Grain size determination for parallel processing , 1988, IEEE Software.

[5]  Vivek Sarkar,et al.  Partitioning and Scheduling Parallel Programs for Multiprocessing , 1989 .

[6]  Frank Thomson Leighton,et al.  Dynamic tree embeddings in butterflies and hypercubes , 1989, SPAA '89.

[7]  Frank D. Anger,et al.  Scheduling Precedence Graphs in Systems with Interprocessor Communication Times , 1989, SIAM J. Comput..

[8]  The LAST Algorithm: A Heuristic-Based Static Task Allocation Algorithm , 1989, ICPP.

[9]  Carolyn McCreary,et al.  Automatic determination of grain size for efficient parallel processing , 1989 .

[10]  Daniel Gajski,et al.  Hypertool: A Programming Aid for Message-Passing Systems , 1990, IEEE Trans. Parallel Distributed Syst..

[11]  Mihalis Yannakakis,et al.  Towards an Architecture-Independent Analysis of Parallel Algorithms , 1990, SIAM J. Comput..

[12]  Hesham El-Rewini,et al.  Scheduling Parallel Program Tasks onto Arbitrary Target Machines , 1990, J. Parallel Distributed Comput..

[13]  William J. Dally,et al.  Network and processor architecture for message-driven computers , 1990 .

[14]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[15]  S. Ranka,et al.  Applications and performance analysis of a compile-time optimization approach for list scheduling algorithms on distributed memory multiprocessors , 1992, Proceedings Supercomputing '92.

[16]  Tao Yang,et al.  Scheduling and code generation for parallel architectures , 1993 .

[17]  Tao Yang,et al.  On the Granularity and Clustering of Directed Acyclic Task Graphs , 1993, IEEE Trans. Parallel Distributed Syst..

[18]  Carolyn McCreary,et al.  A comparison of heuristics for scheduling DAGs on multiprocessors , 1994, Proceedings of 8th International Parallel Processing Symposium.