An energy, performance efficient resource consolidation scheme for heterogeneous cloud datacenters

Abstract Datacenters are the principal electricity consumers for cloud computing that provide an IT backbone for today's business and economy. Numerous studies suggest that most of the servers, in the US datacenters, are idle or less-utilised, making it possible to save energy by using resource consolidation techniques. However, consolidation involves migrations of virtual machines, containers and/or applications, depending on the underlying virtualisation method; that can be expensive in terms of energy consumption and performance loss. In this paper, we: (a) propose a consolidation algorithm which favours the most effective migration among VMs, containers and applications; and (b) investigate how migration decisions should be made to save energy without any negative impact on the service performance. We demonstrate through a number of experiments, using the real workload traces for 800 hosts, approximately 1516 VMs, and more than million containers, how different approaches to migration, will impact on datacenter's energy consumption and performance. We suggest, using reasonable assumptions for datacenter set-up, that there is a trade-off involved between migrating containers and virtual machines. It is more performance efficient to migrate virtual machines; however, migrating containers could be more energy efficient than virtual machines. Moreover, migrating containerised applications, that run inside virtual machines, could lead to energy and performance efficient consolidation technique in large-scale datacenters. Our evaluation suggests that migrating applications could be ~5.5% more energy efficient and ~11.9% more performance efficient than VMs migration. Further, energy and performance efficient consolidation is ~14.6% energy and ~7.9% performance efficient than application migration. Finally, we generalise our results using several repeatable experiments over various workloads, resources and datacenter set-ups.

[1]  Qun Li,et al.  Efficient service handoff across edge servers via docker container migration , 2017, SEC.

[2]  Nicolas Vuillerme,et al.  Software Consolidation as an Efficient Energy and Cost Saving Solution for a SaaS/PaaS Cloud Model , 2015, Euro-Par.

[3]  Weisong Shi,et al.  Energy efficiency comparison of hypervisors , 2019, Sustain. Comput. Informatics Syst..

[4]  T. Baker,et al.  Trusted Energy-Efficient Cloud-Based Services Brokerage Platform , 2015 .

[5]  Muhammad Zakarya,et al.  An Extended Energy-Aware Cost Recovery Approach for Virtual Machine Migration , 2019, IEEE Systems Journal.

[6]  Rajkumar Buyya,et al.  A Framework and Algorithm for Energy Efficient Container Consolidation in Cloud Data Centers , 2015, 2015 IEEE International Conference on Data Science and Data Intensive Systems.

[7]  Lee Gillam.  Will Cloud Gain an Edge, or, CLOSER, to the Edge , 2018, CLOSER.

[8]  Kin K. Leung,et al.  Migrating running applications across mobile edge clouds: poster , 2016, MobiCom.

[9]  Muhammad Zakarya,et al.  Energy and performance aware resource management in heterogeneous cloud datacenters , 2017 .

[10]  Ramakrishnan Rajamony,et al.  An updated performance comparison of virtual machines and Linux containers , 2015, 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[11]  Hai Jin,et al.  Performance and energy modeling for live migration of virtual machines , 2011, HPDC.

[12]  Y. C. Tay,et al.  A Performance Comparison of Containers and Virtual Machines in Workload Migration Context , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems Workshops (ICDCSW).

[13]  Randy H. Katz,et al.  Heterogeneity and dynamicity of clouds at scale: Google trace analysis , 2012, SoCC '12.

[14]  Rajkumar Buyya,et al.  ContainerCloudSim: An environment for modeling and simulation of containers in cloud data centers , 2017, Softw. Pract. Exp..

[15]  Karim Djemame,et al.  Energy-aware cost prediction and pricing of virtual machines in cloud computing environments , 2019, Future Gener. Comput. Syst..

[16]  Yonggang Wen,et al.  Data Center Energy Consumption Modeling: A Survey , 2016, IEEE Communications Surveys & Tutorials.

[17]  K. B. Letaief,et al.  A Survey on Mobile Edge Computing: The Communication Perspective , 2017, IEEE Communications Surveys & Tutorials.

[18]  Fei Huan,et al.  Live Migration of Docker Containers through Logging and Replay , 2015, ICM 2015.

[19]  Karim Djemame,et al.  Energy Prediction for Cloud Workload Patterns , 2016, GECON.

[20]  Ricardo Bianchini,et al.  Resource Central: Understanding and Predicting Workloads for Improved Resource Management in Large Cloud Platforms , 2017, SOSP.

[21]  Shripad Nadgowda,et al.  Voyager: Complete Container State Migration , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS).

[22]  Joel Nider,et al.  Cross-ISA Container Migration , 2016, SYSTOR.

[23]  Richard E. Brown,et al.  United States Data Center Energy Usage Report , 2016 .

[24]  Helen D. Karatza,et al.  Combining containers and virtual machines to enhance isolation and extend functionality on cloud computing , 2019, Future Gener. Comput. Syst..

[25]  Kohei Ichikawa,et al.  Container Rebalancing: Towards Proactive Linux Containers Placement Optimization in a Data Center , 2017, 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC).

[26]  Wei-Mei Chen,et al.  Self-adaptive resource allocation for energy-aware virtual machine placement in dynamic computing cloud , 2018, J. Netw. Comput. Appl..

[27]  Rajkumar Buyya,et al.  OpenStack Neat: a framework for dynamic and energy‐efficient consolidation of virtual machines in OpenStack clouds , 2015, Concurr. Comput. Pract. Exp..

[28]  Liang Tong,et al.  A hierarchical edge cloud architecture for mobile computing , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[29]  Rajkumar Buyya,et al.  CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms , 2011, Softw. Pract. Exp..

[30]  Abhishek Verma,et al.  Large-scale cluster management at Google with Borg , 2015, EuroSys.

[31]  Guillaume Pierre,et al.  An experiment-driven energy consumption model for virtual machine management systems , 2018, Sustain. Comput. Informatics Syst..

[32]  Kin K. Leung,et al.  Live Service Migration in Mobile Edge Clouds , 2017, IEEE Wireless Communications.

[33]  Liang Tong,et al.  Application-aware traffic scheduling for workload offloading in mobile clouds , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[34]  Rizos Sakellariou,et al.  A Cloud Controller for Performance-Based Pricing , 2015, 2015 IEEE 8th International Conference on Cloud Computing.

[35]  Sai Venkat Naresh Kotikalapudi.  Comparing Live Migration between Linux Containers and Kernel Virtual Machine : Investigation study in terms of parameters , 2017 .

[36]  Rajkumar Buyya,et al.  An Energy and Performance Aware Consolidation Technique for Containerized Datacenters , 2019, IEEE Transactions on Cloud Computing.

[37]  Fabien Hermenier,et al.  Scheduling Live Migration of Virtual Machines , 2020, IEEE Transactions on Cloud Computing.

[38]  Zdenek Becvar,et al.  Mobile Edge Computing: A Survey on Architecture and Computation Offloading , 2017, IEEE Communications Surveys & Tutorials.

[39]  Martin J. Kollingbaum,et al.  Efficient Live Migration of Linux Containers , 2018, ISC Workshops.

[40]  Lee Gillam,et al.  Sibling virtual machine co-location confirmation and avoidance tactics for Public Infrastructure Clouds , 2016, The Journal of Supercomputing.

[41]  Adrien Lebre,et al.  Putting the Next 500 VM Placement Algorithms to the Acid Test: The Infrastructure Provider Viewpoint , 2018, IEEE Transactions on Parallel and Distributed Systems.

[42]  Noel De Palma,et al.  Software consolidation as an efficient energy and cost saving solution , 2016, Future Gener. Comput. Syst..

[43]  Brian F. Goldiez,et al.  Combining virtualization and containerization to support interactive games and simulations on the cloud , 2019, Simul. Model. Pract. Theory.

[44]  Wei Gao,et al.  Code offload with least context migration in the mobile cloud , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[45]  Feng Zhao,et al.  Virtual machine power metering and provisioning , 2010, SoCC '10.

[46]  Rajkumar Buyya,et al.  Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in Cloud data centers , 2012, Concurr. Comput. Pract. Exp..

[47]  Mehrdad Dianati,et al.  Exploring edges for connected and autonomous driving , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[48]  Rahim Khan,et al.  H$^2$—A Hybrid Heterogeneity Aware Resource Orchestrator for Cloud Platforms , 2019, IEEE Systems Journal.

[49]  KyoungSoo Park,et al.  CoMon: a mostly-scalable monitoring system for PlanetLab , 2006, OPSR.

[50]  Philippe Merle,et al.  Elasticity in Cloud Computing: State of the Art and Research Challenges , 2018, IEEE Transactions on Services Computing.

[51]  Lizy Kurian John,et al.  Complete System Power Estimation Using Processor Performance Events , 2012, IEEE Transactions on Computers.

[52]  Romain Rouvoy,et al.  Process-level power estimation in VM-based systems , 2015, EuroSys.

[53]  Lee Gillam,et al.  An Energy Aware Cost Recovery Approach for Virtual Machine Migration , 2016, GECON.

[54]  Rahim Khan,et al.  Energy-aware dynamic resource management in elastic cloud datacenters , 2019, Simul. Model. Pract. Theory.

[55]  Lee Gillam,et al.  Performance Evaluation for Cost-Efficient Public Infrastructure Cloud Use , 2014, GECON.

[56]  Chao-Tung Yang,et al.  Virtual machine management system based on the power saving algorithm in cloud , 2017, J. Netw. Comput. Appl..

[57]  Lucas Chaufournier,et al.  Containers and Virtual Machines at Scale: A Comparative Study , 2016, Middleware.

[58]  Radu Prodan,et al.  A Workload-Aware Energy Model for Virtual Machine Migration , 2015, 2015 IEEE International Conference on Cluster Computing.

[59]  César A. F. De Rose,et al.  Server consolidation with migration control for virtualized data centers , 2011, Future Gener. Comput. Syst..