Energy efficiency comparison of hypervisors

Abstract Current cloud data centers are fully virtualized for service consolidation and power/energy reduction. Although virtualization could reduce the real-time power consumption and overall energy consumption, the energy characteristics of hypervisors hosting different workloads have not been well profiled or understood thus far. In this study, we investigate the power and energy characteristics of four mainstream hypervisors and a container engine, namely VMware ESXi, Microsoft Hyper-V, KVM, XenServer, and Docker, on six different platforms (three mainstream 2U rack servers, one emerging ARM64 server, one desktop server, and one laptop) with power measurements made over prolonged periods. We use computation-intensive, memory-intensive, and mixed Web server-database workloads to explore the power and energy characteristics of different hypervisors in order to emulate realistic multi-tenant cloud environments. The results of extensive experiments conducted with four workload levels (very light, light, fair, and very heavy) indicate that the hypervisors exhibit different power and energy characteristics. Our findings are as follows. (1) Hypervisors exhibit different power and energy consumptions on the same hardware running the same workload. (2) Although mainstream hypervisors have different energy efficiencies aligned with different workload types and workload levels, no single hypervisor outperforms the other hypervisors on all platforms in terms of power or energy consumption. (3) Although container virtualization is considered as lightweight virtualization in terms of implementation and maintenance, it is essentially not more power-efficient than conventional virtualization technology. (4) Although the ARM64 server has low power consumption, it completes computation tasks with a long execution time and, sometimes, high energy consumption. Further, ARM64 servers have medium energy consumption per database operation for mixed workloads. The results presented in this paper can provide system designers and data center operators with useful insights for power-aware workload placement and virtual machine scheduling.

[1]  Mor Harchol-Balter,et al.  Optimality analysis of energy-performance trade-off for server farm management , 2010, Perform. Evaluation.

[2]  Israel Koren,et al.  Reducing Energy per Instruction via Dynamic Resource Allocation and Voltage and Frequency Adaptation in Asymmetric Multicores , 2014, 2014 IEEE Computer Society Annual Symposium on VLSI.

[3]  Radu Marculescu,et al.  Sustainability through massively integrated computing: Are we ready to break the energy efficiency wall for single-chip platforms? , 2011, 2011 Design, Automation & Test in Europe.

[4]  Ryan Shea,et al.  Power consumption of virtual machines with network transactions: Measurement and improvements , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[5]  Mahmut T. Kandemir,et al.  Software-Directed Data Access Scheduling for Reducing Disk Energy Consumption , 2012, ICDCS.

[6]  Athanasios V. Vasilakos,et al.  Cloud Computing , 2014, ACM Comput. Surv..

[7]  Li Li,et al.  Data center power minimization with placement optimization of liquid-cooled servers and free air cooling , 2016, Sustain. Comput. Informatics Syst..

[8]  Hsien-Hsin S. Lee,et al.  ATAC: Ambient Temperature-Aware Capping for Power Efficient Datacenters , 2014, SoCC.

[9]  Calton Pu,et al.  Performance Overhead among Three Hypervisors: An Experimental Study Using Hadoop Benchmarks , 2013, 2013 IEEE International Congress on Big Data.

[10]  Ryan Shea,et al.  Energy Efficiency of Cloud Virtual Machines: From Traffic Pattern and CPU Affinity Perspectives , 2017, IEEE Systems Journal.

[11]  Ramakrishnan Rajamony,et al.  An updated performance comparison of virtual machines and Linux containers , 2015, 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[12]  Mingsong Bi,et al.  IAMEM: Interaction-Aware Memory Energy Management , 2013, USENIX Annual Technical Conference.

[13]  Sanjay Ranka,et al.  A genetic algorithm based autotuning approach for performance and energy optimization , 2015, 2015 Sixth International Green and Sustainable Computing Conference (IGSC).

[14]  Prashant J. Shenoy,et al.  Beyond Energy-Efficiency: Evaluating Green Datacenter Applications for Energy-Agility , 2016, ICPE.

[15]  Yonggang Wen,et al.  An Empirical Investigation of the Impact of Server Virtualization on Energy Efficiency for Green Data Center , 2013, Comput. J..

[16]  Thomas F. Wenisch,et al.  CoScale: Coordinating CPU and Memory System DVFS in Server Systems , 2012, 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture.

[17]  Chamath Keppitiyagama,et al.  A performance comparison of hypervisors , 2011 .

[18]  Thu D. Nguyen,et al.  Designing and Managing Data centers Powered by Renewable Energy , 2014, IEEE Micro.

[19]  Timothy Wood,et al.  A component-based performance comparison of four hypervisors , 2013, 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013).

[20]  George Kesidis,et al.  Recouping Energy Costs From Cloud Tenants: Tenant Demand Response Aware Pricing Design , 2015, e-Energy.

[21]  Karthick Rajamani,et al.  Designing Energy-Efficient Servers and Data Centers , 2010, Computer.

[22]  Partha Pratim Pande,et al.  Design of an Energy-Efficient CMOS-Compatible NoC Architecture with Millimeter-Wave Wireless Interconnects , 2013, IEEE Transactions on Computers.

[23]  Miika Komu,et al.  Hypervisors vs. Lightweight Virtualization: A Performance Comparison , 2015, 2015 IEEE International Conference on Cloud Engineering.

[24]  Peter Desnoyers,et al.  Reducing Data Movement Costs Using Energy-Efficient, Active Computation on SSD , 2012, HotPower.

[25]  Ziming Zhang,et al.  Characterizing Power and Energy Usage in Cloud Computing Systems , 2011, 2011 IEEE Third International Conference on Cloud Computing Technology and Science.

[26]  I. Ahmad,et al.  An analysis of disk performance in VMware ESX server virtual machines , 2003, 2003 IEEE International Conference on Communications (Cat. No.03CH37441).

[27]  Roberto Morabito,et al.  Power Consumption of Virtualization Technologies: An Empirical Investigation , 2015, 2015 IEEE/ACM 8th International Conference on Utility and Cloud Computing (UCC).

[28]  Pascal Bouvry,et al.  A holistic model of the performance and the energy efficiency of hypervisors in a high‐performance computing environment , 2014, Concurr. Comput. Pract. Exp..

[29]  Weisong Shi,et al.  Energy efficiency comparison of hypervisors , 2017, 2016 Seventh International Green and Sustainable Computing Conference (IGSC).

[30]  Muli Ben-Yehuda,et al.  Quantitative Comparison of Xen and KVM , 2008 .

[31]  Jian Li,et al.  TAPO: Thermal-aware power optimization techniques for servers and data centers , 2011, 2011 International Green Computing Conference and Workshops.

[32]  Guang R. Gao,et al.  An energy efficient TLB design methodology , 2005, ISLPED '05. Proceedings of the 2005 International Symposium on Low Power Electronics and Design, 2005..

[33]  Li Zhou,et al.  VRAA: virtualized resource auction and allocation based on incentive and penalty , 2012, Cluster Computing.

[34]  Pascal Bouvry,et al.  A Holistic Model of the Performance and the Energy-Efficiency of Hypervisors in an HPC Environment , 2013, EE-LSDS.

[35]  Shaolei Ren,et al.  A New Perspective on Energy Accounting in Multi-Tenant Data Centers , 2016 .

[36]  Yonggang Wen,et al.  Energy efficiency and server virtualization in data centers: An empirical investigation , 2012, 2012 Proceedings IEEE INFOCOM Workshops.

[37]  Jean-Marc Pierson,et al.  Energy-Efficient and Thermal-Aware Resource Management for Heterogeneous Datacenters , 2014, Sustain. Comput. Informatics Syst..

[38]  Rajkumar Buyya,et al.  Energy and Carbon-Efficient Placement of Virtual Machines in Distributed Cloud Data Centers , 2013, Euro-Par.

[39]  Kirk Pruhs,et al.  Cluster before you hallucinate: approximating node-capacitated network design and energy efficient routing , 2014, STOC.

[40]  Yifeng Guo,et al.  Reliability-aware power management for parallel real-time applications with precedence constraints , 2011, 2011 International Green Computing Conference and Workshops.

[41]  Peter Desnoyers,et al.  Active flash: towards energy-efficient, in-situ data analytics on extreme-scale machines , 2013, FAST.

[42]  Pascal Bouvry,et al.  HPC Performance and Energy-Efficiency of Xen, KVM and VMware Hypervisors , 2013, 2013 25th International Symposium on Computer Architecture and High Performance Computing.

[43]  Craig A. Knoblock,et al.  A Survey of Digital Map Processing Techniques , 2014, ACM Comput. Surv..