The Power of Randomized Routing in Heterogeneous Loss Systems

Motivated by cloud computing applications, we consider a multi-server system, consisting of a large number of parallel servers, where jobs arrive according to a Poisson process and are assigned to the servers for processing. Each server has the capacity to process only a finite number of jobs simultaneously and different servers have different capacities. A job is accepted for processing only if there is a vacancy available at the server to which it is assigned. Otherwise, the job is discarded or blocked. We consider randomized schemes to assign jobs to servers with the aim of reducing the average blocking probability of jobs in the system. In particular, we consider a scheme that assigns an incoming job to the server having maximum available vacancy among d randomly sampled servers. We consider the system in the limit where both the number of servers and the arrival rate of jobs are scaled by a large factor. This gives rise to a mean field analysis. We show that in the limiting system servers behave independently. Stationary tail probabilities of server occupancies are obtained from the stationary solution of the mean field which is shown to be unique and globally attractive. We further characterize the rate of decay of the stationary tail probabilities. Numerical results suggest that the proposed scheme significantly reduces the average blocking probability of jobs compared to static schemes that probabilistically route jobs to servers independently of their states.

[1]  Bruce E. Hajek,et al.  The supermarket game , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[2]  Vladimir Marbukh.  Loss circuit switched communication network-performance analysis and dynamic routing , 1993, Queueing Syst. Theory Appl..

[3]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[4]  Michael Mitzenmacher,et al.  The Power of Two Choices in Randomized Load Balancing , 2001, IEEE Trans. Parallel Distributed Syst..

[5]  Klaus Deimling.  COUNTABLE SYSTEMS OF ORDINARY DIFFERENTIAL EQUATIONS , 1977 .

[6]  C. Graham,et al.  Propagation of chaos for a fully connected loss network with alternate routing , 1993 .

[7]  K. Deimling.  Ordinary differential equations in Banach spaces , 1977 .

[8]  R. L. Dobrushin,et al.  Queueing system with selection of the shortest of two queues: an assymptotic approach , 1996 .

[9]  Alexander L. Stolyar,et al.  A large-scale service system with packing constraints: minimizing the number of occupied servers , 2013, SIGMETRICS '13.

[10]  P. Whittle.  Partial balance and insensitivity , 1985, Journal of Applied Probability.

[11]  Anthony Unwin,et al.  Reversibility and Stochastic Networks , 1980 .

[12]  Yu. M. Suhov,et al.  Fast Jackson networks , 1999 .

[13]  R. Srikant,et al.  Power of d Choices for Large-Scale Bin Packing , 2015, SIGMETRICS.

[14]  V. Anantharam.  A Mean Field Limit for a Lattice Caricature of Dynamic Routing in Circuit Switched Networks , 1991 .

[15]  C. Graham,et al.  Stochastic particle approximations for generalized Boltzmann models and convergence estimates , 1997 .

[16]  Ravi Mazumdar,et al.  Rate-based randomized routing in large heterogeneous processor sharing systems , 2014, 2014 26th International Teletraffic Congress (ITC).

[17]  Ravi Mazumdar,et al.  Analysis of Randomized Join-the-Shortest-Queue (JSQ) Schemes in Large Heterogeneous Processor-Sharing Systems , 2016, IEEE Transactions on Control of Network Systems.

[18]  S. Turner,et al.  The Effect of Increasing Routing Choice on Resource Pooling , 1998, Probability in the Engineering and Informational Sciences.

[19]  R. Srikant,et al.  Stochastic models of load balancing and scheduling in cloud computing clusters , 2012, 2012 Proceedings IEEE INFOCOM.