2DFQ: Two-Dimensional Fair Queuing for Multi-Tenant Cloud Services

In many important cloud services, different tenants execute their requests in the thread pool of the same process, requiring fair sharing of resources. However, using fair queue schedulers to provide fairness in this context is difficult because of high execution concurrency, and because request costs are unknown and have high variance. Using fair schedulers like WFQ and WF²Q in such settings leads to bursty schedules, where large requests block small ones for long periods of time. In this paper, we propose Two-Dimensional Fair Queueing (2DFQ), which spreads requests of different costs across di erent threads and minimizes the impact of tenants with unpredictable requests. In evaluation on production workloads from Azure Storage, a large-scale cloud system at Microsoft, we show that 2DFQ reduces the burstiness of service by 1-2 orders of magnitude. On workloads where many large requests compete with small ones, 2DFQ improves 99th percentile latencies by up to 2 orders of magnitude.

[1]  Hui Zhang,et al.  Hierarchical packet fair queueing algorithms , 1996, SIGCOMM '96.

[2]  Banu Özden,et al.  Fair queuing for aggregated multiple links , 2001, SIGCOMM.

[3]  Eric A. Brewer,et al.  USENIX Association Proceedings of HotOS IX : The 9 th Workshop on Hot Topics in Operating Systems , 2003 .

[4]  Werner Vogels,et al.  Dynamo: amazon's highly available key-value store , 2007, SOSP.

[5]  Desmond P. Taylor,et al.  A Generalized Processor Sharing Approach to Flow Control in Integrated Services Networks: The SingleNode Case , 2007 .

[6]  Roger M. Needham,et al.  On the duality of operating system structures , 1979, OPSR.

[7]  Harrick M. Vin,et al.  Start-time fair queueing: a scheduling algorithm for integrated services packet switching networks , 1996, SIGCOMM 1996.

[8]  Rodrigo Fonseca,et al.  Retro: Targeted Resource Management in Multi-tenant Distributed Systems , 2015, NSDI.

[9]  Harrick M. Vin,et al.  A hierarchial CPU scheduler for multimedia operating systems , 1996, OSDI '96.

[10]  Pietro Michiardi,et al.  PSBS: Practical Size-Based Scheduling , 2014, IEEE Transactions on Computers.

[11]  Harrick M. Vin,et al.  A hierarchial CPU scheduler for multimedia operating systems , 1996, OSDI '96.

[12]  George Varghese,et al.  Efficient fair queueing using deficit round robin , 1995, SIGCOMM '95.

[13]  Ju Wang,et al.  Windows Azure Storage: a highly available cloud storage service with strong consistency , 2011, SOSP.

[14]  Scott Shenker,et al.  Analysis and simulation of a fair queueing algorithm , 1989, SIGCOMM 1989.

[15]  Luiz André Barroso,et al.  The tail at scale , 2013, CACM.

[16]  Carl A. Waldspurger,et al.  Lottery and stride scheduling: flexible proportional-share resource management , 1995 .

[17]  Marvin Theimer,et al.  Cooperative Task Management Without Manual Stack Management , 2002, USENIX Annual Technical Conference, General Track.

[18]  Peter J. Varman,et al.  Balancing fairness and efficiency in tiered storage systems with bottleneck-aware allocation , 2014, FAST.

[19]  Peter J. Varman,et al.  pClock: an arrival curve based approach for QoS guarantees in shared storage systems , 2007, SIGMETRICS '07.

[20]  Randy H. Katz,et al.  Cake: enabling high-level SLOs on shared storage systems , 2012, SoCC '12.

[21]  Eduard Ayguadé Parra,et al.  A Survey on Performance Management for Internet Applications , 2010 .

[22]  Eduard Ayguadé,et al.  A survey on performance management for internet applications , 2010 .

[23]  HahnScott,et al.  Efficient and scalable multiprocessor fair scheduling using distributed weighted round-robin , 2009 .

[24]  Salil S. Kanhere,et al.  An evaluation of fair packet schedulers using a novel measure of instantaneous fairness , 2005, Comput. Commun..

[25]  Peter J. Varman,et al.  mClock: Handling Throughput Variability for Hypervisor IO Scheduling , 2010, OSDI.

[26]  George C. Necula,et al.  Capriccio: scalable threads for internet services , 2003, SOSP '03.

[27]  Martin Grund,et al.  Impala: A Modern, Open-Source SQL Engine for Hadoop , 2015, CIDR.

[28]  Tao Yang,et al.  Integrated resource management for cluster-based Internet services , 2002, OSDI.

[29]  S. Rus,et al.  Kudu : Storage for Fast Analytics on Fast Data ∗ , 2016 .

[30]  Abhay Parekh,et al.  A generalized processor sharing approach to flow control in integrated services networks-the single node case , 1992, [Proceedings] IEEE INFOCOM '92: The Conference on Computer Communications.

[31]  Ion Stoica,et al.  Efficient Coflow Scheduling Without Prior Knowledge , 2015, SIGCOMM.

[32]  John K. Ousterhout,et al.  Why Threads Are A Bad Idea , 2013 .

[33]  Antony I. T. Rowstron,et al.  IOFlow: a software-defined storage architecture , 2013, SOSP.

[34]  Carl A. Waldspurger,et al.  Stride Scheduling: Deterministic Proportional- Share Resource Management , 1995 .

[35]  Scott Shenker,et al.  Analysis and simulation of a fair queueing algorithm , 1989, SIGCOMM '89.

[36]  Neil Deshpande,et al.  Analysis of the Go runtime scheduler , 2012 .

[37]  Tong Li,et al.  Efficient and scalable multiprocessor fair scheduling using distributed weighted round-robin , 2009, PPoPP '09.

[38]  Vyas Sekar,et al.  Multi-resource fair queueing for packet processing , 2012, CCRV.

[39]  Mahadev Konar,et al.  ZooKeeper: Wait-free Coordination for Internet-scale Systems , 2010, USENIX ATC.

[40]  Harrick M. Vin,et al.  Start-time fair queueing: a scheduling algorithm for integrated services packet switching networks , 1996, SIGCOMM '96.

[41]  Hui Zhang,et al.  Hierarchical packet fair queueing algorithms , 1996, SIGCOMM 1996.

[42]  QueueingJon,et al.  WF 2 Q : Worst-case Fair Weighted Fair , 1996 .

[43]  Hitesh Ballani,et al.  End-to-end Performance Isolation Through Virtual Datacenters , 2014, OSDI.

[44]  Banu Ozden,et al.  Fair queuing for aggregated multiple links , 2001, SIGCOMM 2001.

[45]  Adam Wierman,et al.  Scheduling despite inexact job-size information , 2008, SIGMETRICS '08.

[46]  William E. Weihl,et al.  Lottery scheduling: flexible proportional-share resource management , 1994, OSDI '94.

[47]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[48]  G. Varghese.  Eecient Fair Queuing Using Deecit round Robin , 1995 .

[49]  Hui Lu,et al.  vFair: latency-aware fair storage scheduling via per-IO cost-based differentiation , 2015, SoCC.

[50]  Hui Zhang,et al.  WF/sup 2/Q: worst-case fair weighted fair queueing , 1996, Proceedings of IEEE INFOCOM '96. Conference on Computer Communications.

[51]  Yingwei Luo,et al.  Failure Recovery: When the Cure Is Worse Than the Disease , 2013, HotOS.

[52]  Michael J. Freedman,et al.  From application requests to virtual IOPs: provisioned key-value storage with Libra , 2014, EuroSys '14.

[53]  Richard Wolski,et al.  Quorum: flexible quality of service for internet services , 2005, NSDI.

[54]  Anees Shaikh,et al.  Performance Isolation and Fairness for Multi-Tenant Cloud Storage , 2012, OSDI.

[55]  Chandandeep Singh Pabla.  Completely fair scheduler , 2009 .