Performance Evaluation of Containers for HPC

Container-based virtualization technologies such as LXC or Docker have gained a lot of interest recently, especially in the HPC context where they could help to address a number of long-running issues. Even if they have proven to perform better than full-fledged, hypervisor-based, virtualization solutions, there are still a lot of questions about the use of container solutions in the HPC context. This paper evaluates the performance of Linux-based container solutions that rely on cgroups and namespaces using the NAS parallel benchmarks, in various configurations. We show that containers technology has matured over the years, and that performance issues are being solved.

[1]  Arnaud Legrand,et al.  Toward Better Simulation of MPI Applications on Ethernet/TCP Networks , 2013, PMBS@SC.

[2]  Ramakrishnan Rajamony,et al.  An updated performance comparison of virtual machines and Linux containers , 2015, 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[3]  Alex Garthwaite,et al.  The evolution of an x86 virtual machine monitor , 2010, OPSR.

[4]  David H. Bailey,et al.  The Nas Parallel Benchmarks , 1991, Int. J. High Perform. Comput. Appl..

[5]  César A. F. De Rose,et al.  Performance Evaluation of Container-Based Virtualization for High Performance Computing Environments , 2013, 2013 21st Euromicro International Conference on Parallel, Distributed, and Network-Based Processing.

[6]  Jeanna Neefe Matthews,et al.  Quantifying the performance isolation properties of virtualization systems , 2007, ExpCS '07.

[7]  Chandra Krintz,et al.  Evaluating the Performance Impact of Xen on MPI and Process Execution For HPC Systems , 2006, First International Workshop on Virtualization Technology in Distributed Computing (VTDC 2006).

[8]  Amar Phanishayee,et al.  Safe and effective fine-grained TCP retransmissions for datacenter communication , 2009, SIGCOMM '09.

[9]  Jaspal Subhlok,et al.  Characterizing NAS benchmark performance on shared heterogeneous networks , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[10]  Nathan Regola,et al.  Recommendations for Virtualization Technologies in High Performance Computing , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[11]  Allen D. Malony,et al.  The Tau Parallel Performance System , 2006, Int. J. High Perform. Comput. Appl..

[12]  Emmanuel Jeanvoine,et al.  Design and Evaluation of a Virtual Experimental Environment for Distributed Systems , 2013, 2013 21st Euromicro International Conference on Parallel, Distributed, and Network-Based Processing.

[13]  Olivier Richard,et al.  Reconstructable Software Appliances with Kameleon , 2015, OPSR.

[14]  John Paul Walters,et al.  A Comparison of Virtualization Technologies for HPC , 2008, 22nd International Conference on Advanced Information Networking and Applications (aina 2008).

[15]  Gabriel Antoniu,et al.  A performance evaluation of Azure and Nimbus clouds for scientific applications , 2012, CloudCP '12.

[16]  Fabienne Anhalt,et al.  Linux-based virtualization for HPC clusters , 2009 .

[17]  Emmanuel Jeannot,et al.  Adding Virtualization Capabilities to the Grid'5000 Testbed , 2012, CLOSER.

[18]  Marty Humphrey,et al.  A quantitative analysis of high performance computing with Amazon's EC2 infrastructure: The death of the local cluster? , 2009, 2009 10th IEEE/ACM International Conference on Grid Computing.