Optimal Control of Distributed Parallel Server Systems Under the Halfin and Whitt Regime

We consider a distributed parallel server system that consists of multiple server pools and a single customer class. We show that the minimum-expected-delay faster-server-first (MED-FSF) routing policy asymptotically minimizes the stationary distribution of the total queue length and the stationary delay probability in the Halfin and Whitt regime. We propose the minimum-expected-delay load-balancing (MED-LB) routing policy to balance the utilizations of all the servers in a distributed system with no unnecessary idling. We show that this policy balances both the long-run and finite-time average utilizations over all the server pools in the Halfin and Whitt regime. We next show that, under either the MED-FSF or the MED-LB policy, a distributed system performs as well as the corresponding inverted V-system. Finally, we show that, operating under the MED-LB policy, both the distributed system and the inverted V-system have similar performances to a corresponding M/M/n system. We illustrate the quality of our asymptotic results for several parallel server systems via simulation experiments.

[1]  J. Michael Harrison,et al.  Heavy traffic resource pooling in parallel‐server systems , 1999, Queueing Syst. Theory Appl..

[2]  Sem C. Borst,et al.  Dimensioning Large Call Centers , 2000, Oper. Res..

[3]  Ward Whitt,et al.  Deciding Which Queue to Join: Some Counterexamples , 1986, Oper. Res..

[4]  A. Stolyar.  MaxWeight scheduling in a generalized switch: State space collapse and workload minimization in heavy traffic , 2004 .

[5]  Tapani Lehtonen,et al.  On the optimality of the shortest line discipline , 1984 .

[6]  Constantinos Maglaras,et al.  On Customer Contact Centers with a Call-Back Option: Customer Decisions, Routing Rules, and System Design , 2004, Oper. Res..

[7]  Avishai Mandelbaum,et al.  Staffing and Control of Large-Scale Service Systems with Multiple Customer Classes and Fully Flexible Servers , 2005 .

[8]  Avishai Mandelbaum,et al.  Designing a Call Center with Impatient Customers , 2002, Manuf. Serv. Oper. Manag..

[9]  Tolga Tezcan,et al.  Dynamic Control of N-Systems with Many Servers: Asymptotic Optimality of a Static Priority Policy in Heavy Traffic , 2010, Oper. Res..

[10]  G. J. Foschini,et al.  A Basic Dynamic Routing Problem and Diffusion , 1978, IEEE Trans. Commun..

[11]  Constantinos Maglaras,et al.  Decision , Risk and Operations Working Papers Series Contact Centers with a CallBack Option and Real-Time Delay Information , 2004 .

[12]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[13]  Ward Whitt,et al.  Heavy-Traffic Limits for the G/H2*/n/mQueue , 2005, Math. Oper. Res..

[14]  J. Dai.  On Positive Harris Recurrence of Multiclass Queueing Networks: A Unified Approach Via Fluid Limit Models , 1995 .

[15]  J. V. Mieghem.  Dynamic Scheduling with Convex Delay Costs: The Generalized CU Rule , 1995 .

[16]  Alexander L. Stolyar,et al.  Scheduling Flexible Servers with Convex Delay Costs: Heavy-Traffic Optimality of the Generalized cµ-Rule , 2004, Oper. Res..

[17]  M. Reiman,et al.  The multiclass GI/PH/N queue in the Halfin-Whitt regime , 2000, Advances in Applied Probability.

[18]  Maury Bramson,et al.  State space collapse with application to heavy traffic limits for multiclass queueing networks , 1998, Queueing Syst. Theory Appl..

[19]  Martin I. Reiman,et al.  Some diffusion approximations with state space collapse , 1984 .

[20]  A. Mandelbaum,et al.  Design , staffing and control of large service systems : The case of a single customer class and multiple server types , 2004 .

[21]  Alexander L. Stolyar,et al.  OPTIMAL ROUTING IN OUTPUT-QUEUED FLEXIBLE SERVER SYSTEMS , 2005, Probability in the Engineering and Informational Sciences.

[22]  Amy R. Ward,et al.  Critical Thresholds for Dynamic Routing in Queueing Networks , 2002, Queueing Syst. Theory Appl..

[23]  Avishai Mandelbaum,et al.  Telephone Call Centers: Tutorial, Review, and Research Prospects , 2003, Manuf. Serv. Oper. Manag..

[24]  Ronald W. Wolff,et al.  Poisson Arrivals See Time Averages , 1982, Oper. Res..

[25]  François Baccelli,et al.  ON QUEUES WITH IMPATIENT CUSTOMERS. , 1981 .

[26]  W. Whitt.  Heavy-Traffic Limits for the G / H ∗ 2 / n / m Queue Ward Whitt , 2005 .

[27]  Ward Whitt,et al.  Heavy-Traffic Limits for Queues with Many Exponential Servers , 1981, Oper. Res..

[28]  C. Laws.  Resource pooling in queueing networks with dynamic routing , 1992, Advances in Applied Probability.

[29]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[30]  D. Jagerman.  Some properties of the erlang loss function , 1974 .

[31]  Yaakov Kogan,et al.  Call routing to distributed queues: Is FIFO really better than MED? , 1997, Telecommun. Syst..

[32]  Patrick T. Harker,et al.  To Sell or Not to Sell , 1999 .

[33]  Mor Armony,et al.  Dynamic Routing in Large-Scale Service Systems with Heterogeneous Servers , 2005, Queueing Syst. Theory Appl..

[34]  D. Gamarnik,et al.  Validity of heavy traffic steady-state approximations in generalized Jackson networks , 2004, math/0410066.

[35]  Ward Whitt,et al.  A Diffusion Approximation for the G/GI/n/m Queue , 2004, Oper. Res..

[36]  David Gamarnik,et al.  Validity of Heavy Traffic Steady-State Approximations in Open Queueing Networks , 2004 .

[37]  A. Puhalskii.  On the Invariance Principle for the First Passage Time , 1994, Math. Oper. Res..

[38]  R. Weber.  On the optimal assignment of customers to parallel servers , 1978, Journal of Applied Probability.