A network aware approach for the scheduling of virtual machine migration during peak loads

Live virtual machine migration can have a major impact on how a cloud system performs, as it consumes significant amounts of network resources such as bandwidth. Migration contributes to an increase in consumption of network resources which leads to longer migration times and ultimately has a detrimental effect on the performance of a cloud computing system. Most industrial approaches use ad-hoc manual policies to migrate virtual machines. In this paper, we propose an autonomous network aware live migration strategy that observes the current demand level of a network and performs appropriate actions based on what it is experiencing. The Artificial Intelligence technique known as Reinforcement Learning acts as a decision support system, enabling an agent to learn optimal scheduling times for live migration while analysing current network traffic demand. We demonstrate that an autonomous agent can learn to utilise available resources when peak loads saturate the cloud network.

[1]  Matthew Caesar,et al.  Walk the line: consistent network updates with bandwidth guarantees , 2012, HotSDN '12.

[2]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[3]  Jun Yan,et al.  A Network-aware Virtual Machine Placement and Migration Approach in Cloud Computing , 2010, 2010 Ninth International Conference on Grid and Cloud Computing.

[4]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[5]  Rajarshi Das,et al.  A Hybrid Reinforcement Learning Approach to Autonomic Resource Allocation , 2006, 2006 IEEE International Conference on Autonomic Computing.

[6]  Onur Mutlu,et al.  Self-Optimizing Memory Controllers: A Reinforcement Learning Approach , 2008, 2008 International Symposium on Computer Architecture.

[7]  Rajkumar Buyya,et al.  NetworkCloudSim: Modelling Parallel Applications in Cloud Simulations , 2011, 2011 Fourth IEEE International Conference on Utility and Cloud Computing.

[8]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[9]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[10]  Akshat Verma,et al.  pMapper: Power and Migration Cost Aware Application Placement in Virtualized Systems , 2008, Middleware.

[11]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[12]  Alex Sim,et al.  Estimating and Forecasting Network Traffic Performance Based on Statistical Patterns Observed in SNMP Data , 2013, MLDM.

[13]  Andrew Warfield,et al.  Live migration of virtual machines , 2005, NSDI.

[14]  Wei Liu,et al.  Adaptive power management using reinforcement learning , 2009, 2009 IEEE/ACM International Conference on Computer-Aided Design - Digest of Technical Papers.

[15]  Sam Devlin,et al.  Multi-Objective Dynamic Dispatch Optimisation using Multi-Agent Reinforcement Learning: (Extended Abstract) , 2016, AAMAS.

[16]  Rajkumar Buyya,et al.  Energy Efficient Resource Management in Virtualized Cloud Data Centers , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[17]  Enda Barrett,et al.  Applying reinforcement learning towards automating resource allocation and application scalability in the cloud , 2013, Concurr. Comput. Pract. Exp..

[18]  Arun Venkataramani,et al.  Black-box and Gray-box Strategies for Virtual Machine Migration , 2007, NSDI.

[19]  Isis Truck,et al.  Using Reinforcement Learning for Autonomic Resource Allocation in Clouds: towards a fully automated workflow , 2011 .

[20]  Enda Barrett,et al.  A reinforcement learning approach for dynamic selection of virtual machines in cloud data centres , 2016, 2016 Sixth International Conference on Innovative Computing Technology (INTECH).

[21]  Alexander Stage,et al.  Network-aware migration control and scheduling of differentiated virtual machine workloads , 2009, 2009 ICSE Workshop on Software Engineering Challenges of Cloud Computing.

[22]  Rajkumar Buyya,et al.  Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in Cloud data centers , 2012, Concurr. Comput. Pract. Exp..

[23]  Lin Li,et al.  An Online Energy Saving Resource Optimization Methodology for Data Center , 2013, J. Softw..

[24]  Pasi Liljeberg,et al.  Energy-Efficient Virtual Machines Consolidation in Cloud Data Centers Using Reinforcement Learning , 2014, 2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing.

[25]  Biswanath Mukherjee,et al.  Heterogeneous bandwidth provisioning for virtual machine migration over SDN-enabled optical networks , 2014, OFC 2014.

[26]  Andy Hopper,et al.  Predicting the Performance of Virtual Machine Migration , 2010, 2010 IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[27]  Peter Corcoran,et al.  Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning , 2017, ArXiv.

[28]  Enda Barrett,et al.  A reinforcement learning approach for the scheduling of live migration from under utilised hosts , 2016, Memetic Computing.

[29]  Enda Barrett,et al.  A Learning Architecture for Scheduling Workflow Applications in the Cloud , 2011, 2011 IEEE Ninth European Conference on Web Services.

[30]  Lucas Chaufournier,et al.  CloudNet: Dynamic Pooling of Cloud Resources by Live WAN Migration of Virtual Machines , 2011, IEEE/ACM Transactions on Networking.

[31]  Michael Anthony Bauer,et al.  Towards adaptive policy-based management , 2010, 2010 IEEE Network Operations and Management Symposium - NOMS 2010.

[32]  Biswanath Mukherjee,et al.  Bandwidth and routing assignment for virtual machine migration in photonic cloud networks , 2013 .

[33]  Ladan Tahvildari,et al.  Adaptive Action Selection in Autonomic Software Using Reinforcement Learning , 2008, Fourth International Conference on Autonomic and Autonomous Systems (ICAS'08).