A lower bound on the queueing delay in resource constrained load balancing

We consider the following distributed service model: jobs with unit mean, general distribution, and independent processing times arrive as a renewal process of rate $\lambda n$, with $0<\lambda<1$, and are immediately dispatched to one of several queues associated with $n$ identical servers with unit processing rate. We assume that the dispatching decisions are made by a central dispatcher endowed with a finite memory, and with the ability to exchange messages with the servers. We study the fundamental resource requirements (memory bits and message exchange rate), in order to drive the expected queueing delay in steady-state of a typical job to zero, as $n$ increases. We develop a novel approach to show that, within a certain broad class of "symmetric" policies, every dispatching policy with a message rate of the order of $n$, and with a memory of the order of $\log n$ bits, results in an expected queueing delay which is bounded away from zero, uniformly as $n\to\infty$.

[1]  Thomas L. Saaty,et al.  Elements of queueing theory , 1961 .

[2]  Tapani Lehtonen,et al.  On the optimality of the shortest line discipline , 1984 .

[3]  John N. Tsitsiklis,et al.  Optimal distributed policies for choosing among multiple servers , 1991, [1991] Proceedings of the 30th IEEE Conference on Decision and Control.

[4]  R. L. Dobrushin,et al.  Queueing system with selection of the shortest of two queues: an assymptotic approach , 1996 .

[5]  Dror G. Feitelson,et al.  Improved Utilization and Responsiveness with Gang Scheduling , 1997, JSSPP.

[6]  Eli Upfal,et al.  Balanced Allocations , 1999, SIAM J. Comput..

[7]  Mor Harchol-Balter,et al.  On Choosing a Task Assignment Policy for a Distributed Server System , 1998, J. Parallel Distributed Comput..

[8]  Michael Mitzenmacher,et al.  The Power of Two Choices in Randomized Load Balancing , 2001, IEEE Trans. Parallel Distributed Syst..

[9]  Devavrat Shah,et al.  Load balancing with memory , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[10]  Mark Burgess,et al.  Dynamic pull-based load balancing for autonomic servers , 2008, NOMS 2008 - 2008 IEEE Network Operations and Management Symposium.

[11]  Yury Makarychev,et al.  Balanced Allocation: Memory Performance Tradeoffs , 2009, ArXiv.

[12]  Noga Alon,et al.  Choice-Memory Tradeoff in Allocations , 2009, 2009 50th Annual IEEE Symposium on Foundations of Computer Science.

[13]  F. Baccelli,et al.  Elements of Queueing Theory: Palm Martingale Calculus and Stochastic Recurrences , 2010 .

[14]  Yi Lu,et al.  Decay of Tails at Equilibrium for FIFO Join the Shortest Queue Networks , 2011, ArXiv.

[15]  James R. Larus,et al.  Join-Idle-Queue: A novel load balancing algorithm for dynamically scalable web services , 2011, Perform. Evaluation.

[16]  Christoph Lenzen,et al.  Tight bounds for parallel randomized load balancing , 2011, Distributed Computing.

[17]  R. Srikant,et al.  The power of slightly more than one sample in randomized load balancing , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[18]  Alexander L. Stolyar Pull-based load distribution in large-scale heterogeneous service systems , 2015, Queueing Syst. Theory Appl..

[19]  Sem C. Borst,et al.  Universality of Power-of-d Load Balancing Schemes , 2016, PERV.

[20]  J. Tsitsiklis,et al.  Delay, Memory, and Messaging Tradeoffs in Distributed Service Systems , 2016, SIGMETRICS.

[21]  Benny Van Houdt,et al.  On the Power-of-d-choices with Least Loaded Server Selection , 2018, Proc. ACM Meas. Anal. Comput. Syst..