Adaptive Scheduling of Parallel Jobs on Functionally Heterogeneous Resources

A parallel program usually incurs operations on multiple processing resources, interleaving computations, I/Os, and communications, where each task can only be executed on a processor of a matching category. Many parallel systems also embed special-purpose processors like vector units, floating-point co-processors, and various I/O processors. Presently, there is no provably good scheduling algorithm that ensures efficient use of multiple resources with functional heterogeneity. This paper presents K-RAD, an algorithm that adoptively schedules parallel jobs on multiple processing resources without requiring prior information about the jobs, such as their release times and parallelism profiles. Let K denote the number of categories of heterogenous resources and Pmax denote the maximum number of processors among all categories. We show that, for any set of jobs with arbitrary release times, K-RAD is (K + 1 - 1/Pmax)- competitive with respect to the makespan. This competitive ratio is provably the best possible for any non-clairvoyant deterministic algorithms for K-resource scheduling. We also show that K-RAD is (4K + 1 - 4K/(|J| + 1))- competitive with respect to the mean response time for any batched job set J. For the special case of K = 1, i.e., scheduling on homogeneous resources, the best existing mean response time bound for online non-clairvoyant algorithm is 2 + radic(3) ap 3.73 proved by Edmonds et al. in STOC'97. We show that K-RAD is 3-competitive with respect to the mean response time when K = 1, which offers the best competitive ratio to date.

[1]  Jeffrey M. Jaffe,et al.  Algorithms for Scheduling Tasks on Unrelated Processors , 1981, JACM.

[2]  Peiyi Tang,et al.  Dynamic Processor Self-Scheduling for General Parallel Nested Loops , 1987, IEEE Trans. Computers.

[3]  Xiaotie Deng,et al.  Non-clairvoyant multiprocessor scheduling of jobs with changing execution characteristics (extended abstract) , 1997, STOC '97.

[4]  Virgílio A. F. Almeida,et al.  Cost-performance analysis of heterogeneity in supercomputer architectures , 1990, Proceedings SUPERCOMPUTING '90.

[5]  Fabián A. Chudak,et al.  Approximation algorithms for precedence-constrained scheduling problems on parallel machines that run at different speeds , 1997, SODA '97.

[6]  Xiaotie Deng,et al.  Preemptive Scheduling of Parallel Jobs on Multiprocessors , 1996, SIAM J. Comput..

[7]  E BlellochGuy,et al.  Provably efficient scheduling for languages with fine-grained parallelism , 1999 .

[8]  Yuxiong He,et al.  Provably Efficient Online Nonclairvoyant Adaptive Scheduling , 2007, IEEE Transactions on Parallel and Distributed Systems.

[9]  Yacine Atif,et al.  Dynamic scheduling techniques for heterogeneous computing systems , 1995, Concurr. Pract. Exp..

[10]  Philip S. Yu,et al.  Scheduling parallel tasks to minimize average response time , 1994, SODA '94.

[11]  David P. Williamson,et al.  Scheduling parallel machines on-line , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[12]  Rajeev Motwani,et al.  Non-clairvoyant scheduling , 1994, SODA '93.

[13]  Veljko M. Milutinovic,et al.  EM³: A Taxonomy of Heterogeneous Computing Systems , 1995, Computer.

[14]  Bruce M. Maggs,et al.  Packet routing and job-shop scheduling inO(congestion+dilation) steps , 1994, Comb..

[15]  Charles E. Leiserson,et al.  Space-Efficient Scheduling of Multithreaded Computations , 1998, SIAM J. Comput..

[16]  Xiaotie Deng,et al.  Competitive dynamic multiprocessor allocation for parallel applications , 1995, Proceedings.Seventh IEEE Symposium on Parallel and Distributed Processing.

[17]  Michael A. Bender,et al.  An Efficient Approximation Algorithm for Minimizing Makespan on Uniformly Related Machines , 1998, IPCO.

[18]  Guy E. Blelloch,et al.  Provably efficient scheduling for languages with fine-grained parallelism , 1999, JACM.

[19]  Daniel A. Reed,et al.  NCSA's World Wide Web Server: Design and Performance , 1995, Computer.

[20]  E.L. Lawler,et al.  Optimization and Approximation in Deterministic Sequencing and Scheduling: a Survey , 1977 .

[21]  David J. Lilja,et al.  Implementing a dynamic processor allocation policy for multiprogrammed parallel applications in the SolarisTM , 2001, Concurr. Comput. Pract. Exp..

[22]  Xiaotie Deng,et al.  On Multiprocessor System Scheduling , 1998, J. Comb. Optim..

[23]  Anoop Gupta,et al.  Process control and scheduling issues for multiprogrammed shared-memory multiprocessors , 1989, SOSP '89.

[24]  Robert D. Blumofe,et al.  Scheduling multithreaded computations by work stealing , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[25]  Edith Schonberg,et al.  Low-overhead scheduling of nested parallelism , 1991, IBM J. Res. Dev..

[26]  Guy E. Blelloch,et al.  A provable time and space efficient implementation of NESL , 1996, ICFP '96.

[27]  Eugene L. Lawler,et al.  Sequencing and scheduling: algorithms and complexity , 1989 .

[28]  Viktor K. Prasanna,et al.  Heterogeneous Supercomputing: Problems and Issues , 1992, Proceedings. Workshop on Heterogeneous Processing.

[29]  David P. Williamson,et al.  Scheduling parallel machines on-line , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[30]  Wednesday September,et al.  2007 International Conference on Parallel Processing , 2007 .

[31]  Philip S. Yu,et al.  Scheduling parallelizable tasks to minimize average response time , 1994, SPAA '94.

[32]  Michael A. Bender,et al.  Scheduling Cilk multithreaded parallel programs on processors of different speeds , 2000, SPAA.

[33]  Raj Vaswani,et al.  A dynamic processor allocation policy for multiprogrammed shared-memory multiprocessors , 1993, TOCS.

[34]  David B. Shmoys,et al.  Improved approximation algorithms for shop scheduling problems , 1991, SODA '91.

[35]  Yuxiong He,et al.  Provably Efficient Two-Level Adaptive Scheduling , 2006, JSSPP.

[36]  Mary K. Vernon,et al.  The performance of multiprogrammed multiprocessor scheduling algorithms , 1990, SIGMETRICS '90.

[37]  Aravind Srinivasan,et al.  Better approximation guarantees for job-shop scheduling , 1997, SODA '97.

[38]  John Zahorjan,et al.  Processor scheduling in shared memory multiprocessors , 1990, SIGMETRICS '90.

[39]  Guy E. Blelloch,et al.  Space-efficient scheduling of nested parallelism , 1999, TOPL.

[40]  Robert D. Blumofe,et al.  Executing multithreaded programs efficiently , 1995 .