The impact of heterogeneity on master-slave scheduling

In this paper, we assess the impact of heterogeneity on scheduling independent tasks on master-slave platforms. We assume a realistic one-port model where the master can communicate with a single slave at any time. We target both on-line and off-line scheduling problems, and we focus on simpler instances where all tasks have the same size. While such on-line problems can be solved in polynomial time on homogeneous platforms, we show that there does not exist any optimal deterministic algorithm for heterogeneous platforms. Whether the source of heterogeneity comes from computation speeds, or from communication bandwidths, or from both, we establish lower bounds on the competitive ratio of any deterministic algorithm. We provide such bounds for the most important objective functions: the minimization of the makespan (or total execution time), the minimization of the maximum response time (difference between completion time and release time), and the minimization of the sum of all response times. Altogether, we obtain nine theorems which nicely assess the impact of heterogeneity on on-line scheduling. For off-line scheduling, we prove several result for problems with release dates, either optimality or NP-hardness. These theoretical contributions are complemented on the practical side by the implementation of several heuristics on a small but fully heterogeneous MPI platform. Our results show the superiority of those heuristics which fully take into account the relative capacity of the communication links.

[1]  Joseph Y.-T. Leung,et al.  Minimizing mean flowtime and makespan on master-slave systems , 2005, J. Parallel Distributed Comput..

[2]  Viktor K. Prasanna,et al.  Efficient collective communication in distributed heterogeneous systems , 2003, J. Parallel Distributed Comput..

[3]  Yves Robert,et al.  A realistic model and an efficient heuristic for scheduling with heterogeneous processors , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[4]  Z Liu,et al.  Scheduling Theory and its Applications , 1997 .

[5]  Yan Alexander Li,et al.  Minimizing the Application Execution Time Through Scheduling of Subtasks and Communication Traffic in a Heterogeneous Computing System , 1997, IEEE Trans. Parallel Distributed Syst..

[6]  Francine Berman,et al.  Overview of the Book: Grid Computing – Making the Global Infrastructure a Reality , 2003 .

[7]  Leonel Sousa,et al.  Communication contention in task scheduling , 2005, IEEE Transactions on Parallel and Distributed Systems.

[8]  Pangfeng Liu,et al.  Broadcast scheduling optimization for heterogeneous cluster systems , 2000, SPAA '00.

[9]  Francine Berman,et al.  Grid Computing: Making the Global Infrastructure a Reality , 2003 .

[10]  Jiri Sgall,et al.  On-line scheduling --- a survey , 1997 .

[11]  Sudipto Guha,et al.  Message Multicasting in Heterogeneous Networks , 2000, SIAM J. Comput..

[12]  Manish Parashar,et al.  Understanding the Behavior and Performance of Non-blocking Communications in MPI , 2004, Euro-Par.

[13]  Samir Khuller,et al.  On broadcasting in heterogenous networks , 2004, SODA '04.

[14]  Tsan-sheng Hsu,et al.  Scheduling Problems in a Practical Allocation Model , 1997, J. Comb. Optim..

[15]  Yves Robert,et al.  The impact of heterogeneity on master-slave on-line scheduling , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[16]  J. M. Moore.  An n Job, One Machine Sequencing Algorithm for Minimizing the Number of Late Jobs , 1968 .

[17]  Arnold L. Rosenberg,et al.  Optimal sharing of bags of tasks in heterogeneous clusters , 2003, SPAA '03.

[18]  Ali R. Hurson,et al.  Scheduling and Load Balancing in Parallel and Distributed Systems , 1995 .

[19]  Fernando Guirado,et al.  Improving static scheduling using inter-task concurrency measures , 2001, Proceedings International Conference on Parallel Processing Workshops.

[20]  Barbara B. Simons,et al.  Multiprocessor Scheduling of Unit-Time Jobs with Arbitrary Release Times and Deadlines , 1983, SIAM J. Comput..

[21]  Michael G. Norman,et al.  Models of machines and computation for mapping in multicomputers , 1993, CSUR.

[22]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[23]  Joseph Y.-T. Leung,et al.  Minimizing sum of completion times and makespan in master-slave systems , 2006, IEEE Transactions on Computers.

[24]  Joseph Y.-T. Leung,et al.  Handbook of Scheduling: Algorithms, Models, and Performance Analysis , 2004 .

[25]  Henri Casanova,et al.  Parameter Sweeps on the Grid with APST , 2003 .

[26]  Pierre-François Dutot,et al.  Complexity of master-slave tasking on heterogeneous trees , 2005, Eur. J. Oper. Res..

[27]  Tsan-sheng Hsu,et al.  Task Allocation on a Network of Processors , 2000, IEEE Trans. Computers.

[28]  H. Ali,et al.  Task Scheduling in Multiprocessing Systems , 1995, Computer.

[29]  Jirí Sgall,et al.  On-line Scheduling , 1996, Online Algorithms.

[30]  Yves Robert,et al.  The master-slave paradigm with heterogeneous processors , 2001, Proceedings 42nd IEEE Symposium on Foundations of Computer Science.

[31]  Dhabaleswar K. Panda,et al.  Efficient collective communication on heterogeneous networks of workstations , 1998, Proceedings. 1998 International Conference on Parallel Processing (Cat. No.98EX205).

[32]  Yves Robert,et al.  Off-line and on-line scheduling on heterogeneous master-slave platforms , 2006, 14th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP'06).

[33]  Dhabaleswar K. Panda,et al.  Communication modeling of heterogeneous networks of workstations for performance characterization of collective operations , 1999, Proceedings. Eighth Heterogeneous Computing Workshop (HCW'99).

[34]  Yves Robert,et al.  A Polynomial-Time Algorithm for Allocating Independent Tasks on Heterogeneous Fork-Graphs , 2002 .

[35]  E.L. Lawler,et al.  Optimization and Approximation in Deterministic Sequencing and Scheduling: a Survey , 1977 .

[36]  Dieter Kranzlmüller.  Scheduling and load balancing , 2003 .