Optimal resource allocation in synchronized multi-tier Internet services

Modern Internet systems have evolved from simple monolithic systems to complex multi-tiered architectures. For these systems, providing good response time performance is crucial for commercial success. In practice, the response-time performance of multi-tiered systems is often degraded by severe synchronization problems, causing jobs to wait for responses from different tiers. Synchronization between different tiers is a complicating factor in the optimal control and analysis of performance. In this paper, we study a generic multi-tier model with synchronization in a queuing-theoretical setting. The system is able to share processing capacity between arriving jobs that need to be sent to other tiers and the responses that have arrived after processing from these tiers. We provide structural results on the optimal resource allocation policy and provide a full characterization of the policy in the framework of Markov decision theory. We also highlight important effects of synchronization in the model. We validate our expressions through extensive experimentations for a wide range of resource configurations.

[1]  Onno Boxma,et al.  Sojourn-time approximations in a two-node queueing network , 2005 .

[2]  Ben Shneiderman,et al.  Response time and display rate in human performance with computers , 1984, CSUR.

[3]  Onno Boxma,et al.  Sojourn times in queueing networks , 1989 .

[4]  J. Morrison.  Response-Time Distribution for a Processor-Sharing System , 1985 .

[5]  Asser N. Tantawi,et al.  An analytical model for multi-tier internet services and its applications , 2005, SIGMETRICS '05.

[6]  B. Urgaonkar,et al.  Cataclysm: policing extreme overloads in internet applications , 2005, WWW '05.

[7]  J. R. Jackson.  Networks of Waiting Lines , 1957 .

[8]  Daniel A. Menascé,et al.  Resource Allocation for Autonomic Data Centers using Analytic Performance Models , 2005, Second International Conference on Autonomic Computing (ICAC'05).

[9]  Robert D. van der Mei,et al.  Modelling End-to-end Quality-of-Service for Transaction-Based Services in Multi-Domain Environments , 2006, 2006 IEEE International Conference on Web Services (ICWS'06).

[10]  Daniel A. Menascé.  Web Server Software Architectures , 2003, IEEE Internet Comput..

[11]  Robert B. Cooper,et al.  An Introduction To Queueing Theory , 2016 .

[12]  Hideaki Takagi,et al.  Stochastic Analysis of Computer and Communication Systems , 1990 .

[13]  Edward G. Coffman,et al.  Waiting Time Distributions for Processor-Sharing Systems , 1970, JACM.

[14]  Dan Rubenstein,et al.  Provisioning servers in the application tier for e-commerce systems , 2004, Twelfth IEEE International Workshop on Quality of Service, 2004. IWQOS 2004..

[15]  T. Ott.  THE SOJOURN-TIME DISTRIBUTION IN THE M/G/1 QUEUE , 1984 .

[16]  Dan Rubenstein,et al.  Provisioning servers in the application tier for e-commerce systems , 2004, IWQoS.

[17]  K. Shin,et al.  Performance Guarantees for Web Server End-Systems: A Control-Theoretical Approach , 2002, IEEE Trans. Parallel Distributed Syst..

[18]  Wei Jin,et al.  USENIX Association Proceedings of USITS ’ 03 : 4 th USENIX Symposium on Internet Technologies and Systems , 2003 .

[19]  Jin Chen,et al.  Autonomic Provisioning of Backend Databases in Dynamic Content Web Servers , 2006, 2006 IEEE International Conference on Autonomic Computing.

[20]  Robert D. van der Mei,et al.  Sojourn time approximations in queueing networks with feedback , 2006, Perform. Evaluation.

[21]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .