QDSL: a queuing model for systems with differential service levels

A feature exhibited by many modern computing systems is their ability to improve the quality of output they generate for a given input by spending more computing resources on processing it. Often this improvement comes at the price of degraded performance in the form of reduced throughput or increased response time. We formulate QDSL, a class of constrained optimization problems defined in the context of a queueing server equipped with multiple levels of service. Solutions to QDSL provide rules for dynamically varying the service level to achieve desired trade-offs between output quality and performance. Our approach involves reducing restricted versions of such systems to Markov Decision Processes. We find two variants of such systems worth studying: (i) VarSL, in which a single request may be serviced using a combination of multiple levels during its lifetime and (ii) FixSL in which the service level may not change during the lifetime of a request. Our modeling indicates that optimal service level selection policies in these systems correspond to very simple rules that can be implemented very efficiently in realistic, online systems. We find our policies to be useful in two response-time-sensitive real-world systems: (i) qSecStore, an iSCSI-based secure storage system that has access to multiple encryption functions, and (ii) qPowServer, a server with DVFS-capable processor. As a representative result, in an instance of qSecStore serving disk requests derived from the well-regarded TPC-H traces, we are able to improve the fraction of requests using more reliable encryption functions by 40-60%, while meeting performance targets. In a simulation of qPowServer employing realistic DVFS parameters, we are able to improve response times significantly while only violating specified server-wide power budgets by less than 5W.

[1]  Erez Zadok,et al.  Proceedings of the General Track: 2003 Usenix Annual Technical Conference Ncryptfs: a Secure and Convenient Cryptographic File System , 2022 .

[2]  Tie-Yan Liu,et al.  Dynamic selection and effective compression of key frames for video abstraction , 2003, Pattern Recognit. Lett..

[3]  Ronald A. Howard,et al.  Dynamic Probabilistic Systems , 1971 .

[4]  Srinivasan Keshav,et al.  A Framework for Differentiated Services , 1999 .

[5]  Shlomo Zilberstein,et al.  Using Anytime Algorithms in Intelligent Systems , 1996, AI Mag..

[6]  Scott Shenker,et al.  Analysis and simulation of a fair queueing algorithm , 1989, SIGCOMM 1989.

[7]  Anand Sivasubramaniam,et al.  Design, implementation and evaluation of security in iSCSI-based network storage systems , 2006, StorageSS '06.

[8]  T. B. Crabill Optimal Control of a Service Facility with Variable Exponential Service Times and Constant Arrival Rate , 1972 .

[9]  E. Feinberg Optimal control of average reward constrained continuous-time finite Markov decision processes , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[10]  Massoud Pedram,et al.  Dynamic power management based on continuous-time Markov decision processes , 1999, DAC '99.

[11]  Alan Jay Smith,et al.  Improving dynamic voltage scaling algorithms with PACE , 2001, SIGMETRICS '01.

[12]  J. Michael Harrison,et al.  Dynamic Control of a Queue with Adjustable Service Rate , 2001, Oper. Res..

[13]  Frank Bellosa,et al.  Process Cruise Control: Throttling Memory Access in a Soft Real-Time Environment , 1997, SOSP 1997.

[14]  William A. Wulf,et al.  TOWARDS A FRAMEWORK FOR SECURITY MEASUREMENT , 1997 .

[15]  Pramod K. Varshney,et al.  QoS Support in Wireless Sensor Networks: A Survey , 2004, International Conference on Wireless Networks.

[16]  S. Wittevrongel,et al.  Queueing Systems , 2019, Introduction to Stochastic Processes and Simulation.

[17]  Harrick M. Vin,et al.  Start-time fair queueing: a scheduling algorithm for integrated services packet switching networks , 1997, TNET.

[18]  Analysis and Simulation of a Fair Queuing Algorithm , 2008 .

[19]  Chandra Krintz,et al.  Reducing delay with dynamic selection of compression formats , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[20]  Marvin A. Sirbu,et al.  Distributed network storage service with quality-of-service guarantees , 2000, J. Netw. Comput. Appl..

[21]  Jonathan Walpole,et al.  Control challenges in multi-level adaptive video streaming , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[22]  Peter Druschel,et al.  Resource containers: a new facility for resource management in server systems , 1999, OSDI '99.

[23]  Wayne D. Smith,et al.  TPC-W: Benchmarking An Ecommerce Solution , 2001 .

[24]  R. Weber,et al.  Optimal control of service rates in networks of queues , 1987, Advances in Applied Probability.

[25]  Nina Bhatti,et al.  Web server support for tiered services , 1999, IEEE Netw..

[26]  Cynthia E. Irvine,et al.  Toward a taxonomy and costing method for security services , 1999, Proceedings 15th Annual Computer Security Applications Conference (ACSAC'99).

[27]  Frank Kelly,et al.  Charging and rate control for elastic traffic , 1997, Eur. Trans. Telecommun..

[28]  E. N. Elnozahy,et al.  Energy Conservation Policies for Web Servers , 2003, USENIX Symposium on Internet Technologies and Systems.

[29]  Erik Riedel,et al.  A Framework for Evaluating Storage System Security , 2002, FAST.

[30]  D. Efrosinin,et al.  Numerical Analysis of Optimal Control Policies for Queueing Systems with Heterogeneous Servers 1 , 2002 .

[31]  Margaret Martonosi,et al.  Data compression algorithms for energy-constrained devices in delay tolerant networks , 2006, SenSys '06.

[32]  Amin Vahdat,et al.  Managing energy and server resources in hosting centers , 2001, SOSP.

[33]  Tao Yu,et al.  Service selection algorithms for Web services with end-to-end QoS constraints , 2004, Proceedings. IEEE International Conference on e-Commerce Technology, 2004. CEC 2004..

[34]  Harrick M. Vin,et al.  Start-time fair queueing: a scheduling algorithm for integrated services packet switching networks , 1996, SIGCOMM '96.

[35]  Kevin Skadron,et al.  Power-aware QoS management in Web servers , 2003, RTSS 2003. 24th IEEE Real-Time Systems Symposium, 2003.

[36]  Linn I. Sennott,et al.  Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs , 1989, Oper. Res..

[37]  B. Ohlman,et al.  A Framework for Differentiated Services , 1998 .

[38]  E. Elnozahy,et al.  Energy Conservation for Servers , 2001 .

[39]  Philip Levis,et al.  Policies for dynamic clock scheduling , 2000, OSDI.

[40]  Sugih Jamin,et al.  A measurement-based admission-controlled Web server , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[41]  Leonard Kleinrock,et al.  Queueing Systems: Volume I-Theory , 1975 .

[42]  Banu Özden,et al.  The Eclipse Operating System: Providing Quality of Service via Reservation Domains , 1998, USENIX ATC.

[43]  Oliver Rose,et al.  Estimation of the Hurst Parameter of Long-Range Dependent Time Series , 1996 .