Recommendations for Virtualization Technologies in High Performance Computing

The benefits of virtualization are typically considered to be server consolidation, (leading to the reduction of power and cooling costs) increased availability, isolation, ease of operating system deployment and simplified disaster recovery. High Performance Computing (HPC) environments pose one main challenge for virtualization: the need to maximize throughput with minimal loss of CPU and I/O efficiency. However, virtualization is usually evaluated in terms of enterprise workloads and assumes that servers are underutilized and can be consolidated. In this paper we evaluate the performance of several virtual machine technologies in the context of HPC. A fundamental requirement of current high performance workloads is that both CPU and I/O must be highly efficient for tasks such as MPI jobs. This work benchmarks two virtual machine monitors, Open VZ and KVM, specifically focusing on I/O throughput since CPU efficiency has been extensively studied [1]. Open VZ offers near native I/O performance. Amazon’s EC2 “ClusterCompute Node” product is also considered for comparative purposes and performs quite well. The EC2 “Cluster ComputeNode” product utilizes the Xen hyper visor in hvm mode and 10Gbit/s Ethernet for high throughput communication. Therefore, we also briefly studied Xen on our hardware platform (in hvmmode) to determine if there are still areas of improvement in KVM that allow EC2 to outperform KVM (with InfiniBand host channel adapters operating at 20 Gbit/s) in MPI benchmarks. We conclude that KVM’s I/O performance is sub optimal, potentially due to memory management problems in the hyper visor. Amazon’sEC2 service is promising, although further investigation is necessary to understand the effects of network based storage on I/O throughput in compute nodes. Amazon’s offering may be attractive for users searching for “InfiniBand-like” performance without the upfront investment required to build an InfiniBand cluster or users wishing to dynamically expand their cluster during periods of high demand.

[1]  Rupak Biswas,et al.  Characterizing Application Performance Sensitivity to R esource Contention in Multicore Architectures , 2009 .

[2]  Gil Neiger,et al.  Intel ® Virtualization Technology for Directed I/O , 2006 .

[3]  Rupak Biswas,et al.  Early performance evaluation of a "Nehalem" cluster using scientific and engineering applications , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[4]  Rusty Russell,et al.  virtio: towards a de-facto standard for virtual I/O devices , 2008, OPSR.

[5]  Renato J. O. Figueiredo,et al.  A case for grid computing on virtual machines , 2003, 23rd International Conference on Distributed Computing Systems, 2003. Proceedings..

[6]  Dhabaleswar K. Panda,et al.  High Performance VMM-Bypass I/O in Virtual Machines , 2006, USENIX Annual Technical Conference, General Track.

[7]  David H. Bailey,et al.  The Nas Parallel Benchmarks , 1991, Int. J. High Perform. Comput. Appl..

[8]  Alan L. Cox,et al.  Optimizing network virtualization in Xen , 2006 .

[9]  Hans Werner Meuer,et al.  Top500 Supercomputer Sites , 1997 .

[10]  John Paul Walters,et al.  A Comparison of Virtualization Technologies for HPC , 2008, 22nd International Conference on Advanced Information Networking and Applications (aina 2008).

[11]  Chandra Krintz,et al.  Evaluating the Performance Impact of Xen on MPI and Process Execution For HPC Systems , 2006, First International Workshop on Virtualization Technology in Distributed Computing (VTDC 2006).

[12]  Fabienne Anhalt,et al.  Linux-based virtualization for HPC clusters , 2009 .

[13]  Kang G. Shin,et al.  Performance Evaluation of Virtualization Technologies for Server Consolidation , 2007 .

[14]  John Paul Walters,et al.  A fault-tolerant strategy for virtualized HPC clusters , 2009, The Journal of Supercomputing.

[15]  Willy Zwaenepoel,et al.  Diagnosing performance overheads in the xen virtual machine environment , 2005, VEE '05.

[16]  Dhabaleswar K. Panda,et al.  A case for high performance computing with virtual machines , 2006, ICS '06.

[17]  Chandra Krintz,et al.  Paravirtualization for HPC Systems , 2006, ISPA Workshops.

[18]  Robert P. Goldberg,et al.  Survey of virtual machine research , 1974, Computer.

[19]  Zhang Jian,et al.  The Optimization of Xen Network Virtualization , 2008, 2008 International Conference on Computer Science and Software Engineering.