Nudge: Stochastically Improving upon FCFS

The First-Come First-Served (FCFS) scheduling policy is the most popular scheduling algorithm used in practice. Furthermore, its usage is theoretically validated: for light-tailed job size distributions, FCFS has weakly optimal asymptotic tail of response time. But what if we don't just care about the asymptotic tail? What if we also care about the 99th percentile of response time, or the fraction of jobs that complete in under one second? Is FCFS still best? Outside of the asymptotic regime, only loose bounds on the tail of FCFS are known, and optimality is completely open. In this paper, we introduce a new policy, Nudge, which is the first policy to provably stochastically improve upon FCFS. We prove that Nudge simultaneously improves upon FCFS at every point along the tail, for light-tailed job size distributions. As a result, Nudge outperforms FCFS for every moment and every percentile of response time. Moreover, Nudge provides a multiplicative improvement over FCFS in the asymptotic tail. This resolves a long-standing open problem by showing that, counter to previous conjecture, FCFS is not strongly asymptotically optimal.

[1]  Konstantin Avrachenkov,et al.  Optimal policy for multi-class scheduling in a single server queue , 2009, 2009 21st International Teletraffic Congress.

[2]  Dejan S. Milojicic,et al.  SLA Decomposition: Translating Service Level Objectives to System Level Thresholds , 2007, Fourth International Conference on Autonomic Computing (ICAC'07).

[3]  Linus Schrage,et al.  Letter to the Editor - A Proof of the Optimality of the Shortest Remaining Processing Time Discipline , 1968, Oper. Res..

[4]  Ward Whitt,et al.  Asymptotics for M/G/1 low-priority waiting-time tail probabilities , 1997, Queueing Syst. Theory Appl..

[5]  Mark M. Davis,et al.  How Long Should a Customer Wait for Service , 1991 .

[6]  Linus Schrage,et al.  The Queue M/G/1 With Feedback to Lower Priority Queues , 1967 .

[7]  Mor Harchol-Balter,et al.  On Choosing a Task Assignment Policy for a Distributed Server System , 1998, J. Parallel Distributed Comput..

[8]  Jing-Sheng Song,et al.  Price, delivery time guarantees and capacity selection , 1998, Eur. J. Oper. Res..

[9]  John T. Blake,et al.  An Analysis Of Emergency Room Wait Time Issues Via Computer Simulation , 1996 .

[10]  Peter D. Welch,et al.  On a Generalized M/G/1 Queuing Process in Which the First Customer of Each Busy Period Receives Exceptional Service , 1964 .

[11]  Peter G. Taylor,et al.  Waiting time distributions in the accumulating priority queue , 2014, Queueing Syst. Theory Appl..

[12]  Adam Wierman,et al.  Preventing Large Sojourn Times Using SMART Scheduling , 2008, Oper. Res..

[13]  G. J. A. Stern,et al.  Queueing Systems, Volume 2: Computer Applications , 1976 .

[14]  Mor Harchol-Balter,et al.  Open problems in queueing theory inspired by datacenter computing , 2021, Queueing Syst. Theory Appl..

[15]  J. Kingman.  Inequalities in the Theory of Queues , 1970 .

[16]  Mor Harchol-Balter.  Task assignment with unknown duration , 2002, JACM.

[17]  Patrick Chareka,et al.  A Finite-Interval Uniqueness Theorem for Bilateral Laplace Transforms , 2007, Int. J. Math. Math. Sci..

[18]  Eric J. Friedman,et al.  Fairness and efficiency in web server protocols , 2003, SIGMETRICS '03.

[19]  Alan Scheller-Wolf,et al.  SOAP: One Clean Analysis of All Age-Based Scheduling Policies , 2017, PERV.

[20]  Adam Wierman,et al.  Is Tail-Optimal Scheduling Possible? , 2012, Oper. Res..

[21]  Ronald W. Wolff,et al.  Poisson Arrivals See Time Averages , 1982, Oper. Res..

[22]  Ward Whitt,et al.  Waiting-time tail probabilities in queues with long-tail service-time distributions , 1994, Queueing Syst. Theory Appl..

[23]  J. F. C. Kingman,et al.  On queues in which customers are served in random order , 1962, Mathematical Proceedings of the Cambridge Philosophical Society.

[24]  Linus Schrage,et al.  The Queue M/G/1 with the Shortest Remaining Processing Time Discipline , 1966, Oper. Res..

[25]  Onno J. Boxma,et al.  Characterizing Policies with Optimal Response Time Tails under Heavy-Tailed Job Sizes , 2020, Proc. ACM Meas. Anal. Comput. Syst..

[26]  Adam Wierman,et al.  Tail-robust scheduling via limited processor sharing , 2010, Perform. Evaluation.

[27]  Timothy L. Urban.  Production , Manufacturing and Logistics Establishing delivery guarantee policies , 2009 .

[28]  Jeffrey C. Mogul,et al.  Nines are Not Enough: Meaningful Metrics for Clouds , 2019, HotOS.

[29]  Ward Whitt,et al.  Exponential Approximations for Tail Probabilities in Queues, I: Waiting Times , 1995, Oper. Res..

[30]  Mor Harchol-Balter.  The Effect of Heavy-Tailed Job Size Distributions on Computer System Design , 1999 .

[31]  Robert B. Cooper,et al.  Stochastic Decompositions in the M/G/1 Queue with Generalized Vacations , 1985, Oper. Res..

[32]  Mor Harchol-Balter,et al.  Borg: the next generation , 2020, EuroSys.

[33]  J. Kingman.  A martingale inequality in the theory of queues , 1964 .

[34]  Percy H. Brill.  A Brief Outline of the Level Crossing Method in Stochastic Models , 2000 .

[35]  Ward Whitt,et al.  Exponential Approximations for Tail Probabilities in Queues II: Sojourn Time and Workload , 1996, Oper. Res..

[36]  Mor Harchol-Balter,et al.  Performance Modeling and Design of Computer Systems: Queueing Theory in Action , 2013 .

[37]  Elizabeth H Bradley,et al.  US emergency department performance on wait time and length of visit. , 2010, Annals of emergency medicine.

[38]  T. Sakurai,et al.  Approximating M/G/1 Waiting Time Tail Probabilities , 2004 .

[39]  M. Crovella,et al.  Heavy-tailed probability distributions in the World Wide Web , 1998 .

[40]  A. Stolyar,et al.  LARGEST WEIGHTED DELAY FIRST SCHEDULING: LARGE DEVIATIONS AND OPTIMALITY , 2001 .