The natural work-stealing algorithm is stable

In this paper we analyse a very simple dynamic work-stealing algorithm. In the work-generation model, there are n generators which are arbitrarily distributed among a set of n processors. During each time-step, with probability /spl lambda/, each generator generates a unit-time task which it inserts into the queue of its host processor. After the new tasks are generated, each processor removes one task from its queue and services it. Clearly, the work-generation model allows the load to grow more and more imbalanced, so, even when /spl lambda/<1, the system load can be unbounded. The natural work-stealing algorithm that we analyse works as follows. During each time step, each empty processor sends a request to a randomly selected other processor. Any non-empty processor having received at least one such request in turn decides (again randomly) in favour of one of the requests. The number of tasks which are transferred from the non-empty processor to the empty one is determined by the so-called work-stealing function f. We analyse the long-term behaviour of the system as a function of /spl lambda/ and f. We show that the system is stable for any constant generation rate /spl lambda/<1 and for a wide class of functions f. We give a quantitative description of the functions f which lead to stable systems. Furthermore, we give upper bounds on the average system load (as a function of f and n).

[1]  Eli Upfal,et al.  Balanced allocations (extended abstract) , 1994, STOC '94.

[2]  Reinhard Lüling,et al.  A dynamic distributed load balancing algorithm with provable good performance , 1993, SPAA '93.

[3]  Ramesh K. Sitaraman,et al.  The power of two random choices: a survey of tech-niques and results , 2001 .

[4]  Michael Mitzenmacher,et al.  On the Analysis of Randomized Load Balancing Schemes , 1997, SPAA '97.

[5]  Leslie Ann Goldberg,et al.  The Natural Work-Stealing Algorithm is Stable , 2001, SIAM J. Comput..

[6]  Robert D. Blumofe,et al.  Scheduling multithreaded computations by work stealing , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[7]  J. Tsitsiklis,et al.  Performance of Multiclass Markovian Queueing Networks Via Piecewise Linear Lyapunov Functions , 2001 .

[8]  G. Fayolle,et al.  Topics in the Constructive Theory of Countable Markov Chains , 1995 .

[9]  Leslie Ann Goldberg,et al.  An Improved Stability Bound for Binary Exponential Backoff , 2001, Theory of Computing Systems.

[10]  Bradley C. Kuszmaul,et al.  Cilk: an efficient multithreaded runtime system , 1995, PPOPP '95.

[11]  Micah Adler,et al.  Analyzing an Infinite Parallel Job Allocation Process , 1998, ESA.

[12]  R. Pemantle,et al.  Moment conditions for a sequence with negative drift to be uniformly bounded in Lr , 1999, math/0404093.

[13]  Eli Upfal,et al.  A simple load balancing scheme for task allocation in parallel machines , 1991, SPAA '91.

[14]  Berthold Vöcking,et al.  Balanced allocations: the heavily loaded case , 2000, STOC '00.

[15]  Burkhard Monien,et al.  Studying overheads in massively parallel MIN/MAX-tree evaluation , 1994, SPAA '94.

[16]  Thomas Decker,et al.  Virtual data space - load balancing for irregular applications , 2000, Parallel Comput..

[17]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[18]  Leslie Ann Goldberg,et al.  Analysis of practical backoff protocols for contention resolution with multiple servers , 1996, SODA '96.

[19]  Paul G. Spirakis,et al.  Scheduling Algorithms for Strict Multithreaded Computations , 1996, ISAAC.

[20]  Petra Berenbrink Parallel Continuous Randomized Load Balancing , 1998 .

[21]  B. Vocking How asymmetry helps load balancing , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[22]  Petra Berenbrink,et al.  Randomized and adversarial load balancing , 1999, SPAA '99.

[23]  M. Mitzenmacher,et al.  Parallel randomized load balancing , 1998 .

[24]  Michael Mitzenmacher,et al.  Load balancing and density dependent jump Markov processes , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[25]  Rajeev Motwani,et al.  Randomized algorithms , 1996, CSUR.

[26]  Michael Mitzenmacher,et al.  Analyses of load stealing models based on differential equations , 1998, SPAA '98.

[27]  Ernst W. Mayr,et al.  Parallel continuous randomized load balancing (extended abstract) , 1998, SPAA '98.

[28]  B. Hajek Hitting-time and occupation-time bounds implied by drift analysis with applications , 1982, Advances in Applied Probability.

[29]  Michael Mitzenmacher,et al.  The Power of Two Choices in Randomized Load Balancing , 2001, IEEE Trans. Parallel Distributed Syst..

[30]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[31]  G. Grimmett,et al.  Probability and random processes , 2002 .

[32]  Philippe Flajolet,et al.  Estimating the multiplicities of conflicts to speed their resolution in multiple access channels , 1987, JACM.

[33]  Alan M. Frieze,et al.  On Balls and Bins with Deletions , 1998, RANDOM.

[34]  Artur Czumaj,et al.  Randomized allocation processes , 1997, Proceedings 38th Annual Symposium on Foundations of Computer Science.

[35]  R. L. Dobrushin,et al.  Queueing system with selection of the shortest of two queues: an assymptotic approach , 1996 .

[36]  Frank Thomson Leighton,et al.  Analysis of Backoff Protocols for Multiple Access Channels , 1996, SIAM J. Comput..