Parallel program execution on a heterogeneous PC cluster using task duplication

We propose to use a duplication based approach in scheduling tasks for a heterogeneous cluster of PCs. In duplication based scheduling, critical tasks are redundantly scheduled to more than one machine in order to reduce the number of inter-task communication operations. The start times of the succeeding tasks are also reduced. The task duplication process is guided given the system heterogeneity in that the critical tasks are scheduled or replicated in faster machines. The algorithm has been implemented in our prototype program parallelization tool for generating MPI code executable on a cluster of Pentium PCs. Our experiments using three applications have indicated that the heterogeneity of a PC cluster, being an inevitable feature, is indeed useful for optimizing the execution of parallel programs.

[1]  Yves Robert,et al.  Parallel Gaussian elimination on an MIMD computer , 1988, Parallel Comput..

[2]  Mihalis Yannakakis,et al.  Towards an Architecture-Independent Analysis of Parallel Algorithms , 1990, SIAM J. Comput..

[3]  T. Fahringer.  Automatic Estimation of Communication Costs for Data Parallel Programs , 1996 .

[4]  Boontee Kruatrachue,et al.  Grain size determination for parallel processing , 1988, IEEE Software.

[5]  Cho-Li Wang,et al.  Viable Approaches To Realizing Single System Image in Multicomputer Clusters , 1997 .

[6]  Michel Cosnard,et al.  Automatic task graph generation techniques , 1995, Proceedings of the Twenty-Eighth Annual Hawaii International Conference on System Sciences.

[7]  Ishfaq Ahmad,et al.  CASCH: A Software Tool for Automatic Parallelization and Scheduling of Programs on Message-Passing Multiprocessors , 1999 .

[8]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[9]  Y.-K. Kwok,et al.  Static scheduling algorithms for allocating directed task graphs to multiprocessors , 1999, CSUR.

[10]  Albert Y. Zomaya,et al.  A Framework for Reinforcement-Based Scheduling in Parallel Processor Systems , 1998, IEEE Trans. Parallel Distributed Syst..

[11]  Edward A. Lee,et al.  A Compile-Time Scheduling Heuristic for Interconnection-Constrained Heterogeneous Processor Architectures , 1993, IEEE Trans. Parallel Distributed Syst..

[12]  R. F. Freund,et al.  Guest Editor's Introduction: Heterogeneous Processing , 1993 .

[13]  Ishfaq Ahmad,et al.  Benchmarking and Comparison of the Task Graph Scheduling Algorithms , 1999, J. Parallel Distributed Comput..

[14]  Michael G. Norman,et al.  Models of machines and computation for mapping in multicomputers , 1993, CSUR.

[15]  Hesham H. Ali,et al.  Task scheduling in parallel and distributed systems , 1994, Prentice Hall series in innovative technology.

[16]  John K. Antonio,et al.  Software support for heterogeneous computing , 1996, CSUR.

[17]  Jing-Chiou Liou,et al.  Task Clustering and Scheduling for Distributed Memory Parallel Architectures , 1996, IEEE Trans. Parallel Distributed Syst..

[18]  Behrooz Shirazi,et al.  DFRN: a new approach for duplication based scheduling for distributed memory multiprocessor systems , 1997, Proceedings 11th International Parallel Processing Symposium.

[19]  Behrooz Shirazi,et al.  Comparative study of task duplication static scheduling versus clustering and non-clustering techniques , 1995, Concurr. Pract. Exp..

[20]  Ishfaq Ahmad,et al.  Dynamic Critical-Path Scheduling: An Effective Technique for Allocating Task Graphs to Multiprocessors , 1996, IEEE Trans. Parallel Distributed Syst..

[21]  Ishfaq Ahmad,et al.  On Exploiting Task Duplication in Parallel Program Scheduling , 1998, IEEE Trans. Parallel Distributed Syst..

[22]  Vivek Sarkar,et al.  Partitioning and Scheduling Parallel Programs for Multiprocessing , 1989 .

[23]  Daniel Gajski,et al.  Hypertool: A Programming Aid for Message-Passing Systems , 1990, IEEE Trans. Parallel Distributed Syst..

[24]  KwokYu-Kwong,et al.  Static scheduling algorithms for allocating directed task graphs to multiprocessors , 1999 .

[25]  Tao Yang,et al.  List Scheduling With and Without Communication Delays , 1993, Parallel Comput..