The Power of Two Choices in Randomized Load Balancing

We consider the following natural model: customers arrive as a Poisson stream of rate /spl lambda/n, /spl lambda/<1, at a collection of n servers. Each customer chooses some constant d servers independently and uniformly at random from the n servers and waits for service at the one with the fewest customers. Customers are served according to the first-in first-out (FIFO) protocol and the service time for a customer is exponentially distributed with mean 1. We call this problem the supermarket model. We wish to know how the system behaves and in particular we are interested in the effect that the parameter d has on the expected time a customer spends in the system in equilibrium. Our approach uses a limiting, deterministic model representing the behavior as n/spl rarr//spl infin/ to approximate the behavior of finite systems. The analysis of the deterministic model is interesting in its own right. Along with a theoretical justification of this approach, we provide simulations that demonstrate that the method accurately predicts system behavior, even for relatively small systems. Our analysis provides surprising implications. Having d=2 choices leads to exponential improvements in the expected time a customer spends in the system over d=1, whereas having d=3 choices is only a constant factor better than d=2. We discuss the possible implications for system design.

[1]  J. Davenport.  Editor , 1960 .

[2]  N E Manos,et al.  Stochastic Models , 1960, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[3]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[4]  Kazuoki Azuma.  WEIGHTED SUMS OF CERTAIN DEPENDENT RANDOM VARIABLES , 1967 .

[5]  T. Kurtz.  Solutions of ordinary differential equations as limits of pure jump markov processes , 1970, Journal of Applied Probability.

[6]  T. Kurtz.  Limit theorems for sequences of jump Markov processes approximating ordinary differential processes , 1971, Journal of Applied Probability.

[7]  Benjamin L. Schwartz,et al.  Queuing Models with Lane Selection: A New Class of Problems , 1974, Oper. Res..

[8]  Sheldon M. Ross,et al.  Introduction to probability models , 1975 .

[9]  Leonard Kleinrock,et al.  Queueing Systems: Volume I-Theory , 1975 .

[10]  A. Barbour.  Networks of queues and the method of stages , 1976, Advances in Applied Probability.

[11]  Wayne L. Winston.  OPTIMALITY OF THE SHORTEST LINE DISCIPLINE , 1977 .

[12]  Wayne L. Winston.  Optimality of the shortest line discipline , 1977, Journal of Applied Probability.

[13]  Norman L. Johnson,et al.  Urn models and their application , 1977 .

[14]  K. Deimling.  Ordinary differential equations in Banach spaces , 1977 .

[15]  S. Ross.  Average delay in queues with non-stationary Poisson arrivals , 1978, Journal of Applied Probability.

[16]  A. Michel,et al.  Stability theory for countably infinite systems of differential equations , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[17]  R. Weber.  On the optimal assignment of customers to parallel servers , 1978, Journal of Applied Probability.

[18]  T. Kurtz.  Strong approximation theorems for density dependent Markov chains , 1978 .

[19]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[20]  Anthony Unwin,et al.  Reversibility and Stochastic Networks , 1980 .

[21]  M. Vidyasagar,et al.  Qualitative Analysis of Large Scale Dynamical Systems , 2012, IEEE Transactions on Systems, Man, and Cybernetics.

[22]  Gaston H. Gonnet,et al.  Expected Length of the Longest Probe Sequence in Hash Code Searching , 1981, JACM.

[23]  M. Sipser,et al.  Maximum matching in sparse random graphs , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[24]  Richard M. Karp,et al.  Maximum Matchings in Sparse Random Graphs , 1981, FOCS 1981.

[25]  R. Abraham,et al.  Manifolds, Tensor Analysis, and Applications , 1983 .

[26]  Harold J. Kushner,et al.  Approximation and Weak Convergence Methods for Random Processes , 1984 .

[27]  Linda Green,et al.  A Queueing System with General-Use and Limited-Use Servers , 1985, Oper. Res..

[28]  Edward D. Lazowska,et al.  A Comparison of Receiver-Initiated and Sender-Initiated Adaptive Load Sharing , 1986, Perform. Evaluation.

[29]  Edward D. Lazowska,et al.  Adaptive load sharing in homogeneous distributed systems , 1986, IEEE Transactions on Software Engineering.

[30]  W. Grassman.  Approximation and Weak Convergence Methods for Random Processes with Applications to Stochastic Systems Theory (Harold J. Kushner) , 1986 .

[31]  S. Zhou,et al.  A Trace-Driven Simulation Study of Dynamic Load Balancing , 1987, IEEE Trans. Software Eng..

[32]  T. Kurtz.  Approximation of Population Processes , 1987 .

[33]  B. Hajek.  Asymptotic analysis of an assignment problem arising in a distributed communications protocol , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[34]  Songnian Zhou.  A Trace-Driven Simulation Study of Dynamic Load Balancing , 1988, IEEE Trans. Software Eng..

[35]  R. Abraham,et al.  Manifolds, tensor analysis, and applications: 2nd edition , 1988 .

[36]  Edward D. Lazowska,et al.  The limited performance benefits of migrating active processes for load sharing , 1988, SIGMETRICS '88.

[37]  Donald F. Towsley,et al.  Analysis of the Effects of Delays on Load Sharing , 1989, IEEE Trans. Computers.

[38]  Ronald W. Wolff,et al.  Stochastic Modeling and the Theory of Queues , 1989 .

[39]  Donald F. Towsley,et al.  Adaptive Load Sharing in Heterogeneous Distributed Systems , 1990, J. Parallel Distributed Comput..

[40]  Richard M. Karp,et al.  An optimal algorithm for on-line bipartite matching , 1990, STOC '90.

[41]  The Influence of Different Workload Descriptions on a Heuristic Load Balancing Scheme , 1991, IEEE Trans. Software Eng..

[42]  Eli Upfal,et al.  A simple load balancing scheme for task allocation in parallel machines , 1991, SPAA '91.

[43]  Ivo J. B. F. Adan,et al.  Analysis of the symmetric shortest queue problem , 1990 .

[44]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[45]  J. Shanthikumar,et al.  Extremal properties of the FIFO discipline in queueing networks , 1992, Journal of Applied Probability.

[46]  Friedhelm Meyer auf der Heide,et al.  Efficient PRAM simulation on a distributed memory machine , 1992, STOC '92.

[47]  Randolph D. Nelson,et al.  The mathematics of product form queuing networks , 1993, CSUR.

[48]  Friedhelm Meyer auf der Heide,et al.  Simple, efficient shared memory simulations , 1993, SPAA '93.

[49]  Bala Kalyanasundaram,et al.  On-Line Load Balancing of Temporary Tasks , 1993, J. Algorithms.

[50]  Alan M. Frieze,et al.  On the satisfiability and maximum satisfiability of random 3-CNF formulas , 1993, SODA '93.

[51]  Yossi Matias,et al.  An optical simulation of shared memory , 1994, SPAA '94.

[52]  Ivo J. B. F. Adan,et al.  Upper and lower bounds for the waiting time in the symmetric shortest queue system , 1994, Ann. Oper. Res..

[53]  Michael Mitzenmacher,et al.  Bounds on the greedy routing algorithm for array networks , 1994, SPAA '94.

[54]  Eli Upfal,et al.  Balanced allocations (extended abstract) , 1994, STOC '94.

[55]  Micah Adler,et al.  Parallel randomized load balancing , 1995, STOC '95.

[56]  Tail Bounds for Occupancy and the Satisfiability Threshold Conjecture , 1995, Random Struct. Algorithms.

[57]  Yuval Rabani,et al.  Fairness in scheduling , 1995, SODA '95.

[58]  David A. Patterson,et al.  Serverless network file systems , 1995, SOSP.

[59]  Mor Harchol-Balter,et al.  Bounding delays in packet-routing networks , 1995, STOC '95.

[60]  Alan M. Frieze,et al.  Balanced Allocations for Tree-Like Inputs , 1995, Inf. Process. Lett..

[61]  N. Wormald.  Differential Equations for Random Processes and Random Graphs , 1995 .

[62]  Yuval Rabani,et al.  A computational view of population genetics , 1995, STOC '95.

[63]  Ben Atkinson,et al.  A Compensation Approach for Queueing Problems. , 1996 .

[64]  Joel H. Spencer,et al.  Sudden Emergence of a Giantk-Core in a Random Graph , 1996, J. Comb. Theory, Ser. B.

[65]  Volker Stemann,et al.  Parallel balanced allocations , 1996, SPAA '96.

[66]  R.M. Karp, M. Luby, F. Meyer auf der Hei.  Efficient PRAM simulation on a distributed memory machine , 1996 .

[67]  Michael Mitzenmacher,et al.  Load balancing and density dependent jump Markov processes , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[68]  M. Mandjes.  Large Deviations for Performance Analysis , 1996 .

[69]  Jose Renato Santos,et al.  Design of the Rio (randomized I/o) Storage Server , 1997 .

[70]  Michael Mitzenmacher,et al.  Constant time per edge is optimal on rooted tree networks , 1996, SPAA '96.

[71]  Azer Bestavros.  Load profiling: a methodology for scheduling real-time tasks in a distributed system , 1997, Proceedings of 17th International Conference on Distributed Computing Systems.

[72]  Michael Molloy,et al.  The analysis of a list-coloring algorithm on a random graph , 1997, Proceedings 38th Annual Symposium on Foundations of Computer Science.

[73]  Daniel A. Spielman,et al.  Practical loss-resilient codes , 1997, STOC '97.

[74]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[75]  Rajmohan Rajaraman,et al.  On contention resolution protocols and associated probabilistic phenomena , 1998, JACM.

[76]  Yuval Rabani,et al.  A computational view of population genetics , 1995, Symposium on the Theory of Computing.

[77]  S. Wittevrongel,et al.  Queueing Systems , 2019, Introduction to Stochastic Processes and Simulation.

[78]  C. Leiserson,et al.  Scheduling multithreaded computations by work stealing , 1999, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[79]  Michael Mitzenmacher,et al.  On the Analysis of Randomized Load Balancing Schemes , 1997, SPAA '97.

[80]  Eli Upfal,et al.  Balanced Allocations , 1999, SIAM J. Comput..

[81]  Ron Goldman,et al.  Poisson approximation , 2000, Proceedings Geometric Modeling and Processing 2000. Theory and Applications.

[82]  Michael Mitzenmacher,et al.  How Useful Is Old Information? , 2000, IEEE Trans. Parallel Distributed Syst..

[83]  Ramesh K. Sitaraman,et al.  The power of two random choices: a survey of tech-niques and results , 2001 .