A reinforcement learning approach for dynamic selection of virtual machines in cloud data centres

In recent years Machine Learning techniques have proven to reduce energy consumption when applied to cloud computing systems. Reinforcement Learning provides a promising solution for the reduction of energy consumption, while maintaining a high quality of service for customers. We present a novel single agent Reinforcement Learning approach for the selection of virtual machines, creating a new energy efficiency practice for data centres. Our dynamic Reinforcement Learning virtual machine selection policy learns to choose the optimal virtual machine to migrate from an over-utilised host. Our experiment results show that a learning agent has the abilities to reduce energy consumption and decrease the number of migrations when compared to a state-of-the-art approach.

[1]  Akshat Verma,et al.  pMapper: Power and Migration Cost Aware Application Placement in Virtualized Systems , 2008, Middleware.

[2]  Rajkumar Buyya,et al.  Energy-aware resource allocation heuristics for efficient management of data centers for Cloud computing , 2012, Future Gener. Comput. Syst..

[3]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[4]  Richard E. Brown,et al.  Report to Congress on Server and Data Center Energy Efficiency: Public Law 109-431 , 2008 .

[5]  Enda Barrett,et al.  Applying reinforcement learning towards automating resource allocation and application scalability in the cloud , 2013, Concurr. Comput. Pract. Exp..

[6]  Wei Liu,et al.  Adaptive power management using reinforcement learning , 2009, 2009 IEEE/ACM International Conference on Computer-Aided Design - Digest of Technical Papers.

[7]  W. Cleveland.  Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[8]  J. Koomey.  Worldwide electricity used in data centers , 2008 .

[9]  Rajkumar Buyya,et al.  Adaptive threshold-based approach for energy-efficient consolidation of virtual machines in cloud data centers , 2010, MGC '10.

[10]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[11]  Luiz André Barroso,et al.  The Case for Energy-Proportional Computing , 2007, Computer.

[12]  Michael Anthony Bauer,et al.  Towards adaptive policy-based management , 2010, 2010 IEEE Network Operations and Management Symposium - NOMS 2010.

[13]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[14]  Isis Truck,et al.  Using Reinforcement Learning for Autonomic Resource Allocation in Clouds: towards a fully automated workflow , 2011 .

[15]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[16]  Rajarshi Das,et al.  A Hybrid Reinforcement Learning Approach to Autonomic Resource Allocation , 2006, 2006 IEEE International Conference on Autonomic Computing.

[17]  Lin Li,et al.  An Online Energy Saving Resource Optimization Methodology for Data Center , 2013, J. Softw..

[18]  Pasi Liljeberg,et al.  Energy-Efficient Virtual Machines Consolidation in Cloud Data Centers Using Reinforcement Learning , 2014, 2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing.

[19]  Xiaoyun Zhu,et al.  1000 Islands: Integrated Capacity and Workload Management for the Next Generation Data Center , 2008, 2008 International Conference on Autonomic Computing.

[20]  Rajkumar Buyya,et al.  Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in Cloud data centers , 2012, Concurr. Comput. Pract. Exp..