VCONF: a reinforcement learning approach to virtual machines auto-configuration

Virtual machine (VM) technology enables multiple VMs to share resources on the same host. Resources allocated to the VMs should be re-configured dynamically in response to the change of application demands or resource supply. Because VM execution involves privileged domain and VM monitor, this causes uncertainties in VMs' resource to performance mapping and poses challenges in online determination of appropriate VM configurations. In this paper, we propose a reinforcement learning (RL) based approach, namely VCONF, to automate the VM configuration process. VCONF employs model-based RL algorithms to address the scalability and adaptability issues in applying RL in systems management. Experimental results on both controlled environments and a testbed of clouds with Xen VMs and representative server workloads demonstrate the effectiveness of VCONF. The approach is able to find optimal (near optimal) configurations in small scale systems and shows good adaptability and scalability.

[1]  Rajarshi Das,et al.  On the use of hybrid reinforcement learning for autonomic resource allocation , 2007, Cluster Computing.

[2]  David Levine,et al.  Managing Power Consumption and Performance of Computing Systems Using Reinforcement Learning , 2007, NIPS.

[3]  Cheng-Zhong Xu,et al.  A Self-tuning Fuzzy Control Approach for End-to-End QoS Guarantees in Web Servers , 2005, IWQoS.

[4]  Xiaoyun Zhu,et al.  Triage: performance isolation and differentiation for storage systems , 2004, Twelfth IEEE International Workshop on Quality of Service, 2004. IWQOS 2004..

[5]  Mona Attariyan,et al.  AutoBash: improving configuration management with operating system causality analysis , 2007, SOSP.

[6]  Alan L. Cox,et al.  Scheduling I/O in virtual machine monitors , 2008, VEE '08.

[7]  Steven D. Gribble,et al.  Configuration Debugging as Search: Finding the Needle in the Haystack , 2004, OSDI.

[8]  Andrew Warfield,et al.  Live migration of virtual machines , 2005, NSDI.

[9]  Armando Fox,et al.  Ensembles of models for automated diagnosis of system performance problems , 2005, 2005 International Conference on Dependable Systems and Networks (DSN'05).

[10]  Kang G. Shin,et al.  Adaptive control of virtualized resources in utility computing environments , 2007, EuroSys '07.

[11]  Peter Stone,et al.  CARVE: A Cognitive Agent for Resource Value Estimation , 2008, 2008 International Conference on Autonomic Computing.

[12]  Armando Fox,et al.  Capturing, indexing, clustering, and retrieving system history , 2005, SOSP '05.

[13]  George Candea,et al.  Autonomous recovery in componentized Internet applications , 2006, Cluster Computing.

[14]  Onur Mutlu,et al.  Self-Optimizing Memory Controllers: A Reinforcement Learning Approach , 2008, 2008 International Symposium on Computer Architecture.

[15]  G ShinKang,et al.  Adaptive control of virtualized resources in utility computing environments , 2007 .

[16]  Gerald Tesauro,et al.  Online Resource Allocation Using Decompositional Reinforcement Learning , 2005, AAAI.

[17]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[18]  Cheng-Zhong Xu,et al.  A Reinforcement Learning Approach to Online Web Systems Auto-configuration , 2009, 2009 29th IEEE International Conference on Distributed Computing Systems.

[19]  Jeffrey Casazza,et al.  Redefining Server Performance Characterization for Virtualization Benchmarking , 2006 .

[20]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[21]  Lui Sha,et al.  Online response time optimization of Apache web server , 2003, IWQoS'03.

[22]  Richard S. Sutton,et al.  Generalization in ReinforcementLearning : Successful Examples UsingSparse Coarse , 1996 .

[23]  Erich M. Nahum,et al.  Yaksha: a self-tuning controller for managing the performance of 3-tiered Web sites , 2004, Twelfth IEEE International Workshop on Quality of Service, 2004. IWQOS 2004..

[24]  Ashraf Aboulnaga,et al.  Automatic virtual machine configuration for database workloads , 2008, SIGMOD Conference.

[25]  Amin Vahdat,et al.  Enforcing Performance Isolation Across Virtual Machines in Xen , 2006, Middleware.

[26]  Christopher G. Atkeson,et al.  A comparison of direct and model-based reinforcement learning , 1997, Proceedings of International Conference on Robotics and Automation.

[27]  Cheng-Zhong Xu,et al.  Online Measurement of the Capacity of Multi-Tier Websites Using Hardware Performance Counters , 2008, 2008 The 28th International Conference on Distributed Computing Systems.