A Reinforcement Learning Approach to Online Web Systems Auto-configuration

In a web system, configuration is crucial to the performance and service availability. It is a challenge, not only because of the dynamics of Internet traffic, but also the dynamic virtual machine environment the system tends to be run on. In this paper, we propose a reinforcement learning approach for autonomic configuration and reconfiguration of multi-tier web systems. It is able to adapt performance parameter settings not only to the change of workload, but also to the change of virtual machine configurations. The RL approach is enhanced with an efficient initialization policy to reduce the learning time for online decision. The approach is evaluated using TPC-W benchmark on a three-tier website hosted on a Xen-based virtual machine environment. Experiment results demonstrate that the approach can autoconfigure the web system dynamically in response to the change in both workload and VM resource. It can drive the system into a near-optimal configuration setting in less than 25 trial-and-error iterations.

[1]  Cheng-Zhong Xu,et al.  Service migration in distributed virtual machines for adaptive grid computing , 2005, 2005 International Conference on Parallel Processing (ICPP'05).

[2]  Takayuki Osogami,et al.  Optimizing system configurations quickly by guessing at the performance , 2007, SIGMETRICS '07.

[3]  Sven Graupner,et al.  Making the Utility Data Center A Power Station for the Enterprise Grid , 2003 .

[4]  Rajarshi Das,et al.  On the use of hybrid reinforcement learning for autonomic resource allocation , 2007, Cluster Computing.

[5]  Kang G. Shin,et al.  Adaptive control of virtualized resources in utility computing environments , 2007, EuroSys '07.

[6]  Ran Gilad-Bachrach,et al.  Workstation capacity tuning using reinforcement learning , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[7]  Archana Ganapathi,et al.  Why Do Internet Services Fail, and What Can Be Done About It? , 2002, USENIX Symposium on Internet Technologies and Systems.

[8]  Le Yi Wang,et al.  VCONF: a reinforcement learning approach to virtual machines auto-configuration , 2009, ICAC '09.

[9]  Gerald Tesauro,et al.  Online Resource Allocation Using Decompositional Reinforcement Learning , 2005, AAAI.

[10]  Wei Zheng,et al.  Automatic configuration of internet services , 2007, EuroSys '07.

[11]  David Levine,et al.  Managing Power Consumption and Performance of Computing Systems Using Reinforcement Learning , 2007, NIPS.

[12]  Bowei Xi,et al.  A smart hill-climbing algorithm for application server configuration , 2004, WWW '04.

[13]  I-Hsin Chung,et al.  Automated cluster-based Web service performance tuning , 2004, Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004..

[14]  Yan Zhang,et al.  Automatic Performance Tuning for J2EE Application Server Systems , 2005, WISE.

[15]  Peter Stone,et al.  CARVE: A Cognitive Agent for Resource Value Estimation , 2008, 2008 International Conference on Autonomic Computing.

[16]  Onur Mutlu,et al.  Self-Optimizing Memory Controllers: A Reinforcement Learning Approach , 2008, 2008 International Symposium on Computer Architecture.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Lui Sha,et al.  Online response time optimization of Apache web server , 2003, IWQoS'03.

[19]  Ashraf Aboulnaga,et al.  Automatic virtual machine configuration for database workloads , 2008, SIGMOD Conference.