Adaptive Markov‐based approach for dynamic virtual machine consolidation in cloud data centers with quality‐of‐service constraints

Dynamic virtual machine (VM) consolidation is one of the emerging technologies that has been considered for low‐cost computing in cloud data centers. Quality‐of‐service (QoS) assurance is one of the challenging issues in the VM consolidation problem since it is directly affected by the increase of resource utilization due to the consolidations. In this paper, we take advantage of Markov chain models to propose a novel approach for VM consolidation that can be used to explicitly set a desired level of QoS constraint in a data center to ensure the QoS goals while improving system utilization. For this purpose, an energy‐efficient and QoS‐aware best fit decreasing algorithm for VM placement is proposed, which considers QoS objective when determining the location of a migrating VM. This algorithm employs an online transition matrix estimator method to deal with the nonstationary nature of real workload data. We also propose new policies for detecting overloaded and underloaded hosts. The performance of our proposed algorithms is evaluated through simulations. The results show that the proposed VM consolidation algorithms in this paper outperforms the benchmark algorithms in terms of energy consumption, service‐level agreement violations, and other cost factors.

[1]  Sheldon M. Ross,et al.  Introduction to probability models , 1975 .

[2]  C. Paige,et al.  Computation of the stationary distribution of a markov chain , 1975 .

[3]  Sheldon M. Ross,et al.  Introduction to Probability Models (4th ed.). , 1990 .

[4]  KyoungSoo Park,et al.  CoMon: a mostly-scalable monitoring system for PlanetLab , 2006, OPSR.

[5]  Fred Spiring,et al.  Introduction to Statistical Quality Control , 2007, Technometrics.

[6]  Wolf-Dietrich Weber,et al.  Power provisioning for a warehouse-sized computer , 2007, ISCA '07.

[7]  Feng Zhao,et al.  Energy aware consolidation for cloud computing , 2008, CLUSTER 2008.

[8]  Gargi Dasgupta,et al.  Server Workload Analysis for Power Minimization using Consolidation , 2009, USENIX Annual Technical Conference.

[9]  Ian Lumb,et al.  A Taxonomy and Survey of Cloud Computing Systems , 2009, 2009 Fifth International Joint Conference on INC, IMS and IDC.

[10]  Martin Bichler,et al.  A Mathematical Programming Approach for Server Consolidation Problems in Virtualized Data Centers , 2010, IEEE Transactions on Services Computing.

[11]  Rajkumar Buyya,et al.  Energy-Efficient Management of Data Center Resources for Cloud Computing: A Vision, Architectural Elements, and Open Challenges , 2010, PDPTA.

[12]  Albert Y. Zomaya,et al.  Energy efficient utilization of resources in cloud computing systems , 2010, The Journal of Supercomputing.

[13]  Rajkumar Buyya,et al.  CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms , 2011, Softw. Pract. Exp..

[14]  Jie Liu,et al.  Cuanta: quantifying effects of shared on-chip resource interference for consolidated virtual machines , 2011, SoCC.

[15]  Kevin Skadron,et al.  Bubble-up: Increasing utilization in modern warehouse scale computers via sensible co-locations , 2011, 2011 44th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[16]  Rajkumar Buyya,et al.  Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in Cloud data centers , 2012, Concurr. Comput. Pract. Exp..

[17]  Bharadwaj Veeravalli,et al.  Utilization-based pricing for power management and profit optimization in data centers , 2012, J. Parallel Distributed Comput..

[18]  Christine Morin,et al.  Snooze: A Scalable and Autonomic Virtual Machine Management Framework for Private Clouds , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[19]  Jesús Carretero,et al.  E-mc2: A formal framework for energy modelling in cloud computing , 2013, Simul. Model. Pract. Theory.

[20]  Christina Delimitrou,et al.  QoS-Aware scheduling in heterogeneous datacenters with paragon , 2013, TOCS.

[21]  Yi Zhuang,et al.  Constraint Programming based Virtual Cloud Resources Allocation Model , 2013 .

[22]  Christina Delimitrou,et al.  Paragon: QoS-aware scheduling for heterogeneous datacenters , 2013, ASPLOS '13.

[23]  Jan Weglarz,et al.  DCworms - A tool for simulation of energy efficiency in distributed computing infrastructures , 2013, Simul. Model. Pract. Theory.

[24]  Ramin Yahyapour,et al.  QoS-Aware VM Placement in Multi-domain Service Level Agreements Scenarios , 2013, 2013 IEEE Sixth International Conference on Cloud Computing.

[25]  Rajkumar Buyya,et al.  Managing Overloaded Hosts for Dynamic Consolidation of Virtual Machines in Cloud Data Centers under Quality of Service Constraints , 2013, IEEE Transactions on Parallel and Distributed Systems.

[26]  Abbas Horri,et al.  Novel resource allocation algorithms to performance and energy efficiency in cloud computing , 2014, The Journal of Supercomputing.

[27]  Mahmoud Al-Ayyoub,et al.  CloudExp: A comprehensive cloud computing experimental framework , 2014, Simul. Model. Pract. Theory.

[28]  Christina Delimitrou,et al.  Quasar: resource-efficient and QoS-aware cluster management , 2014, ASPLOS.

[29]  Maolin Tang,et al.  A Hybrid Genetic Algorithm for the Energy-Efficient Virtual Machine Placement Problem in Data Centers , 2014, Neural Processing Letters.

[30]  Rajkumar Buyya,et al.  SLA-based virtual machine management for heterogeneous workloads in a cloud datacenter , 2014, J. Netw. Comput. Appl..

[31]  Yefu Wang,et al.  Performance-controlled server consolidation for virtualized data centers with multi-tier applications , 2014, Sustain. Comput. Informatics Syst..

[32]  Liang Liu,et al.  Service level agreement based energy-efficient resource management in cloud data centers , 2014, Comput. Electr. Eng..

[33]  Sunilkumar S. Manvi,et al.  Resource management for Infrastructure as a Service (IaaS) in cloud computing: A survey , 2014, J. Netw. Comput. Appl..

[34]  Saeed Sharifian,et al.  Novel energy and SLA efficient resource management heuristics for consolidation of virtual machines in cloud data centers , 2015, Comput. Electr. Eng..

[35]  Rajkumar Buyya,et al.  OpenStack Neat: a framework for dynamic and energy‐efficient consolidation of virtual machines in OpenStack clouds , 2015, Concurr. Comput. Pract. Exp..

[36]  Mohsen Sharifi,et al.  A New Approach for Dynamic Virtual Machine Consolidation in Cloud Data Centers , 2015 .

[37]  P. Santhi Thilagam,et al.  Heuristics based server consolidation with residual resource defragmentation in cloud data centers , 2015, Future Gener. Comput. Syst..

[38]  Hannu Tenhunen,et al.  Using Ant Colony System to Consolidate VMs for Green Cloud Computing , 2015, IEEE Transactions on Services Computing.

[39]  Jie Wu,et al.  Burstiness-Aware Resource Reservation for Server Consolidation in Computing Clouds , 2016, IEEE Transactions on Parallel and Distributed Systems.

[40]  Abolfazl Toroghi Haghighat,et al.  Energy-aware framework with Markov chain-based parallel simulated annealing algorithm for dynamic management of virtual machines in cloud data centers , 2017, The Journal of Supercomputing.

[41]  Guofeng Zhu,et al.  Energy-efficient and QoS-aware model based resource consolidation in cloud data centers , 2017, Cluster Computing.

[42]  Hassan Taheri,et al.  Novel fuzzy multi objective DVFS-aware consolidation heuristics for energy and SLA efficient resource management in cloud data centers , 2017, J. Netw. Comput. Appl..

[43]  Amir Masoud Rahmani,et al.  Server consolidation techniques in virtualized data centers of cloud environments: A systematic literature review , 2018, Softw. Pract. Exp..

[44]  Abolfazl Toroghi Haghighat,et al.  A stochastic process-based server consolidation approach for dynamic workloads in cloud data centers , 2018, The Journal of Supercomputing.

[45]  Amir Masoud Rahmani,et al.  Self-adaptive architecture for virtual machines consolidation based on probabilistic model evaluation of data centers in Cloud computing , 2018, Cluster Computing.

[46]  Roberto Tagliaferri,et al.  Data Mining: Accuracy and Error Measures for Classification and Prediction , 2019, Encyclopedia of Bioinformatics and Computational Biology.

[47]  Christina Delimitrou,et al.  PARTIES: QoS-Aware Resource Partitioning for Multiple Interactive Services , 2019, ASPLOS.

[48]  Surafel Lemma Abebe,et al.  Energy-aware VM placement algorithms for the OpenStack Neat consolidation framework , 2019, Journal of Cloud Computing.

[49]  Abolfazl Toroghi Haghighat,et al.  Adaptive Markov-based approach for dynamic virtual machine consolidation in cloud data centers with quality-of-service constraints , 2020, Softw. Pract. Exp..