Improving virtual machine I/O performance in cloud environments via I/O functionality offloading

Bamunavita Gamage, Sahan Ph.D., Purdue University, December 2013. Improving Virtual Machine I/O Performance in Cloud Environments via I/O Functionality O↵loading. Major Professors: Dongyan Xu and Ramana Rao Kompella. In recent years there has been a rapid growth in the adoption of cloud computing. A key technology that drives cloud computing is virtualization. In addition to enabling multi-tenancy in cloud environments, virtualizing hosts in the cloud environments has made management of hardware resources increasingly flexible. An important consequence of virtualizing hosts in the cloud is the negative impact it has on the I/O performance of the applications running in the virtual machines. In this dissertation, we address the important problem of alleviating the negative impact of virtualization on I/O of the virtual machines. First, we show that virtual machine scheduling negatively impacts the TCP throughput in virtual machines and o✏oading TCP congestion control functionality to the virtual machine monitor layer can significantly improve the TCP throughput on the transmit path. Second, we discuss a lightweight system, which o✏oads selected TCP functionality to the virtual machine monitor layer to improve TCP throughput for full-duplex TCP connections in virtual machines. This system called vPRO integrates the aforementioned congestion control o✏oading system and an existing system, which o✏oads the TCP acknowledgment functionality to the virtual machine monitor layer. Finally, we show that, when a cloud application running in a virtual machine moves data from one device to another without any transformations, o✏oading the entire I/O processing to the virtual machine monitor layer can avoid most of the virtualization associated overheads and hence improve the I/O performance of the application.

[1]  Xiaolan Zhang,et al.  XenSocket: A High-Throughput Interdomain Transport for Virtual Machines , 2007, Middleware.

[2]  Willy Zwaenepoel,et al.  Diagnosing performance overheads in the xen virtual machine environment , 2005, VEE '05.

[3]  Daniel Gooch,et al.  Communications of the ACM , 2011, XRDS.

[4]  Ada Gavrilovska,et al.  Differential virtual time (DVT): rethinking I/O service differentiation for virtual machines , 2010, SoCC '10.

[5]  Jin-Soo Kim,et al.  Inter-domain socket communications supporting high performance and full binary compatibility on Xen , 2008, VEE '08.

[6]  Steven Hand,et al.  Satori: Enlightened Page Sharing , 2009, USENIX Annual Technical Conference.

[7]  Dhabaleswar K. Panda,et al.  Virtual machine aware communication libraries for high performance computing , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[8]  Arun Venkataramani,et al.  Black-box and Gray-box Strategies for Virtual Machine Migration , 2007, NSDI.

[9]  Greg J. Regnier,et al.  TCP onloading for data center servers , 2004, Computer.

[10]  Beng-Hong Lim,et al.  Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor , 2001, USENIX Annual Technical Conference, General Track.

[11]  Irfan Ahmad,et al.  vIC: Interrupt Coalescing for Virtual Machine Storage Device IO , 2011, USENIX Annual Technical Conference.

[12]  Cho-Li Wang,et al.  vBalance: using interrupt load balance to improve I/O performance for SMP virtual machines , 2012, SoCC '12.

[13]  Willy Zwaenepoel,et al.  TwinDrivers: semi-automatic derivation of fast and safe hypervisor network drivers from guest OS drivers , 2009, ASPLOS.

[14]  Ajay Gulati,et al.  hClock: hierarchical QoS for packet scheduling in a hypervisor , 2013, EuroSys '13.

[15]  Cheng Jin,et al.  FAST TCP: Motivation, Architecture, Algorithms, Performance , 2006, IEEE/ACM Transactions on Networking.

[16]  Alex Landau,et al.  Towards exitless and efficient paravirtual I/O , 2012, SYSTOR '12.

[17]  Peter J. Varman,et al.  Demand Based Hierarchical QoS Using Storage Resource Pools , 2012, USENIX Annual Technical Conference.

[18]  Andrew Warfield,et al.  Live migration of virtual machines , 2005, NSDI.

[19]  Amar Phanishayee,et al.  Safe and effective fine-grained TCP retransmissions for datacenter communication , 2009, SIGCOMM '09.

[20]  Nick McKeown,et al.  OpenFlow: enabling innovation in campus networks , 2008, CCRV.

[21]  Alan L. Cox,et al.  Hyper-Switch: A Scalable Software Virtual Switching Architecture , 2013, USENIX Annual Technical Conference.

[22]  Jeffrey C. Mogul,et al.  TCP Offload Is a Dumb Idea Whose Time Has Come , 2003, HotOS.

[23]  Cong Xu,et al.  vSlicer: latency-aware virtual machine scheduling via differentiated-frequency CPU slicing , 2012, HPDC '12.

[24]  Of references. , 1966, JAMA.

[25]  Muli Ben-Yehuda,et al.  IsoStack - Highly Efficient Network Processing on Dedicated Cores , 2010, USENIX Annual Technical Conference.

[26]  Tal Garfinkel,et al.  A Virtual Machine Introspection Based Architecture for Intrusion Detection , 2003, NDSS.

[27]  Hui Lu,et al.  vTurbo: Accelerating Virtual Machine I/O Processing Using Designated Turbo-Sliced Core , 2013, USENIX Annual Technical Conference.

[28]  Injong Rhee,et al.  CUBIC: a new TCP-friendly high-speed TCP variant , 2008, OPSR.

[29]  Ardalan Kangarlou-Haghighi Improving the reliability and performance of virtual cloud infrastructures , 2011 .

[30]  Peter J. Varman,et al.  mClock: Handling Throughput Variability for Hypervisor IO Scheduling , 2010, OSDI.

[31]  Jian Wang,et al.  XenLoop: a transparent high performance inter-VM network loopback , 2008, HPDC '08.

[32]  Laxmi N. Bhuyan,et al.  Performance characterization and cache-aware core scheduling in a virtualized multi-core server under 10GbE , 2009, 2009 IEEE International Symposium on Workload Characterization (IISWC).

[33]  Alex Landau,et al.  ELI: bare-metal performance for I/O virtualization , 2012, ASPLOS XVII.

[34]  Albert G. Greenberg,et al.  Data center TCP (DCTCP) , 2010, SIGCOMM '10.

[35]  Radu Rugina,et al.  Software Techniques for Avoiding Hardware Virtualization Exits , 2012, USENIX Annual Technical Conference.

[36]  Garth R. Goodson,et al.  Fido: Fast Inter-Virtual-Machine Communication for Enterprise Appliances , 2009, USENIX ATC.

[37]  Alan L. Cox,et al.  Optimizing network virtualization in Xen , 2006 .

[38]  A. Fox,et al.  Cloudstone : Multi-Platform , Multi-Language Benchmark and Measurement Tools for Web 2 . 0 , 2008 .

[39]  Amin Vahdat,et al.  Enforcing Performance Isolation Across Virtual Machines in Xen , 2006, Middleware.

[40]  George Varghese,et al.  Difference engine , 2010, OSDI.

[41]  Radu Sion,et al.  Enhancement of Xen's scheduler for MapReduce workloads , 2011, HPDC '11.

[42]  Muli Ben-Yehuda,et al.  vIOMMU: Efficient IOMMU Emulation , 2011, USENIX Annual Technical Conference.

[43]  Zhao Yu,et al.  SR-IOV Networking in Xen: Architecture, Design and Implementation , 2008, Workshop on I/O Virtualization.

[44]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[45]  Ramana Rao Kompella,et al.  vSnoop: Improving TCP Throughput in Virtualized Environments via Acknowledgement Offload , 2010, 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis.

[46]  Ming Zhang,et al.  Understanding data center traffic characteristics , 2010, CCRV.

[47]  Martín Casado,et al.  Extending Networking into the Virtualization Layer , 2009, HotNets.

[48]  Anand Sivasubramaniam,et al.  Xen and co.: communication-aware CPU scheduling for consolidated xen-based hosting platforms , 2007, VEE '07.

[49]  Randy H. Katz,et al.  Above the Clouds: A Berkeley View of Cloud Computing , 2009 .

[50]  Ramana Rao Kompella,et al.  Opportunistic flooding to improve TCP transmit performance in virtualized clouds , 2011, SOCC '11.

[51]  Mendel Rosenblum,et al.  I/O Virtualization , 2011 .

[52]  Irfan Ahmad,et al.  PARDA: Proportional Allocation of Resources for Distributed Storage Access , 2009, FAST.

[53]  Albert G. Greenberg,et al.  The nature of data center traffic: measurements & analysis , 2009, IMC '09.

[54]  A. K. Choudhury,et al.  Dynamic queue length thresholds for shared-memory packet switches , 1998, TNET.

[55]  Ada Gavrilovska,et al.  On disk I/O scheduling in virtual machines , 2010 .

[56]  Muli Ben-Yehuda,et al.  SplitX: Split Guest/Hypervisor Execution on Multi-Core , 2011, WIOV.

[57]  Willy Zwaenepoel,et al.  Optimizing TCP Receive Performance , 2008, USENIX ATC.

[58]  J. Bottomley,et al.  VirtFS — A virtualization aware File System pass-through , 2010 .

[59]  Alan L. Cox,et al.  Scheduling I/O in virtual machine monitors , 2008, VEE '08.

[60]  Carl A. Waldspurger,et al.  Memory resource management in VMware ESX server , 2002, OSDI '02.

[61]  Larry L. Peterson,et al.  TCP Vegas: End to End Congestion Avoidance on a Global Internet , 1995, IEEE J. Sel. Areas Commun..

[62]  Ole Agesen,et al.  A comparison of software and hardware techniques for x86 virtualization , 2006, ASPLOS XII.

[63]  Denis Filimonov,et al.  VAMOS: Virtualization Aware Middleware , 2011, WIOV.