Dynamic Thread Count Adaptation for Multiple Services in SMP Environments

We propose a dynamic mechanism, thread count adaptation, that adjusts the thread counts that are allocated to services for adapting to CPU requirement variations in SMP environments. Our goal is to increase the maximum throughput available on a system that has multiple dynamic content services while meeting different service time criteria for these services in dynamic workloads. Our challenge is to significantly improve response times for dynamic content on a busy well-tuned thread-pool-based system without prioritizing any specific services. Our experiments demonstrate that a prototype using our approach on J2EE middleware quickly (around every 20 ms) adjusted the thread counts for the services and that it improved the average 90th-percentile response times by up to 27% (and 22% on average) for the SPECjAppServer2004 benchmark.

[1]  David E. Culler,et al.  SEDA: an architecture for well-conditioned, scalable internet services , 2001, SOSP.

[2]  John Zic,et al.  Auto-tune design and evaluation on staged event-driven architecture , 2006, MODDM '06.

[3]  Pedro Furtado,et al.  Deadline and Throughput-Aware Control for Request Processing Systems , 2007, ISPA.

[4]  Cheng-Zhong Xu,et al.  eQoS: Provisioning of Client-Perceived End-to-End QoS Guarantees in Web Servers , 2006, IEEE Transactions on Computers.

[5]  Erich M. Nahum,et al.  A method for transparent admission control and request scheduling in e-commerce web sites , 2004, WWW '04.

[6]  Erich M. Nahum,et al.  Yaksha: a self-tuning controller for managing the performance of 3-tiered Web sites , 2004, Twelfth IEEE International Workshop on Quality of Service, 2004. IWQOS 2004..

[7]  Steve Vinoski,et al.  Object Interconnections Comparing Alternative Programming Techniques for Multi-threaded CORBA Servers : Thread Pool ( Column 6 ) , 1996 .

[8]  Stephen M. Fontes,et al.  WebSphere connector architecture evolution , 2004, IBM Syst. J..

[9]  Tao Yang,et al.  Demand-driven service differentiation in cluster-based network servers , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[10]  John Zic,et al.  Explicitly Controlling the Fair Service for Busy Web Servers , 2007, 2007 Australian Software Engineering Conference (ASWEC'07).

[11]  Dan Tsafrir,et al.  The context-switch overhead inflicted by hardware interrupts (and the enigma of do-nothing loops) , 2007, ExpCS '07.

[12]  Douglas M. Freimuth,et al.  Kernel Mechanisms for Service Differentiation in Overloaded Web Servers , 2001, USENIX Annual Technical Conference, General Track.

[13]  Mor Harchol-Balter,et al.  Web servers under overload: How scheduling can help , 2006, TOIT.

[14]  Tarek F. Abdelzaher,et al.  Web Content Adaptation to Improve Server Overload Behavior , 1999, Comput. Networks.

[15]  Tao Yang,et al.  Request-Aware Scheduling for Busy Internet Services , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[16]  B. Urgaonkar,et al.  Cataclysm: policing extreme overloads in internet applications , 2005, WWW '05.

[17]  Dongping Xu.  Performance Study and Dynamic Optimization Design for Thread Pool Systems , 2004 .

[18]  David E. Culler,et al.  USENIX Association Proceedings of USITS ’ 03 : 4 th USENIX Symposium on Internet Technologies and Systems , 2003 .

[19]  Xiaola Lin,et al.  Analysis of optimal thread pool size , 2000, OPSR.

[20]  Pedro Furtado,et al.  Fair Load-Balancing on Parallel Systems for QoS , 2007, 2007 International Conference on Parallel Processing (ICPP 2007).

[21]  Jordi Torres,et al.  Differentiated Quality of Service for e-Commerce Applications through Connection Scheduling based on System-Level Thread Priorities , 2007, 15th EUROMICRO International Conference on Parallel, Distributed and Network-Based Processing (PDP'07).

[22]  Tao Yang,et al.  Selective early request termination for busy internet services , 2006, WWW '06.

[23]  Douglas C. Schmidt,et al.  Evaluating and Optimizing Thread Pool Strategies for Real-Time CORBA , 2001, OM '01.

[24]  Gerald C. Gannod,et al.  A Self-healing Web Server Using Differentiated Services , 2006, ICSOC.

[25]  Prashant J. Shenoy,et al.  Dynamic resource allocation for shared data centers using online measurements , 2003, IWQoS'03.

[26]  Mor Harchol-Balter,et al.  Connection Scheduling in Web Servers , 1999, USENIX Symposium on Internet Technologies and Systems.

[27]  K. Shin,et al.  Performance Guarantees for Web Server End-Systems: A Control-Theoretical Approach , 2002, IEEE Trans. Parallel Distributed Syst..

[28]  Richard Wolski,et al.  Quorum: flexible quality of service for internet services , 2005, NSDI.

[29]  Douglas C. Schmidt,et al.  Object Interconnections Comparing Alternative Programming Techniques for Multi-threaded CORBA Servers (Column 7) , 1996 .

[30]  Tao Yang,et al.  Integrated resource management for cluster-based Internet services , 2002, OSDI.

[31]  Lei Zhang,et al.  WMQ: Towards A Fine-grained QoS Control for E-Business Servers , 2007, IEEE International Conference on e-Business Engineering (ICEBE'07).

[32]  Shigeru Chiba,et al.  Aspect-oriented application-level scheduling for J2EE servers , 2007, AOSD.

[33]  Mor Harchol-Balter,et al.  Size-based scheduling to improve web performance , 2003, TOCS.

[34]  Sang Hyuk Son,et al.  Feedback Control Architecture and Design Methodology for Service Delay Guarantees in Web Servers , 2006, IEEE Transactions on Parallel and Distributed Systems.