Assessing the Computation and Communication Overhead of Linux Containers for HPC Applications

Virtualization technology provides features that are desirable for high-performance computing (HPC), such as enhanced reproducibility for scientific experiments and flexible execution environment customization. This paper explores the performance implications of applying Operating System (OS) containers in HPC applications. Docker and Singularity are compared to a native baseline with no virtualization, using synthetic workloads and an earthquake simulator called Ondes3D as benchmarks. Our evaluation using up to 256 cores indicate that (a) Singularity containers have a minor performance overhead, (b) Docker containers do suffer from increased network latency, and (c) performance gains are attainable with an optimized container on top of a regular OS.

[1]  Emmanuel Jeanvoine,et al.  Kadeploy3: Efficient and Scalable Operating System Provisioning for HPC Clusters , 2012 .

[2]  David H. Bailey,et al.  The Nas Parallel Benchmarks , 1991, Int. J. High Perform. Comput. Appl..

[3]  Larry L. Peterson,et al.  Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors , 2007, EuroSys '07.

[4]  Jean Roman,et al.  High-performance finite-element simulations of seismic wave propagation in three-dimensional nonlinear inelastic geological media , 2010, Parallel Comput..

[5]  Doina Bein,et al.  Distributed MPI cluster with Docker Swarm mode , 2017, 2017 IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC).

[6]  Edward David Moreno,et al.  Performance Analysis of LXC for HPC Environments , 2015, 2015 Ninth International Conference on Complex, Intelligent, and Software Intensive Systems.

[7]  Violeta Holmes,et al.  Orchestrating Docker Containers in the HPC Environment , 2015, ISC.

[8]  Emmanuel Jeannot,et al.  Adding Virtualization Capabilities to the Grid'5000 Testbed , 2012, CLOSER.

[9]  F. Dupros,et al.  Investigation of historical earthquake by seismic wave propagation simulation : Source parameters of the 1887 M 6 . 3 Ligurian , north-western Italy , earthquake , 2011 .

[10]  Vanessa Sochat,et al.  Singularity: Scientific containers for mobility of compute , 2017, PloS one.

[11]  Nicholas Wells,et al.  BusyBox: A Swiss Army Knife for Linux , 2000 .

[12]  Raj Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[13]  Nam Thoai,et al.  Using Docker in high performance computing applications , 2016, 2016 IEEE Sixth International Conference on Communications and Electronics (ICCE).

[14]  Emily Le,et al.  Performance Analysis of Applications using Singularity Container on SDSC Comet , 2017, PEARC.

[15]  Lucas Mello Schnorr,et al.  Using Simulation to Evaluate and Tune the Performance of Dynamic Load Balancing of an Over-Decomposed Geophysics Application , 2017, Euro-Par.

[16]  Ramakrishnan Rajamony,et al.  An updated performance comparison of virtual machines and Linux containers , 2015, 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[17]  Emmanuel Jeanvoine,et al.  Performance Evaluation of Containers for HPC , 2015, Euro-Par Workshops.

[18]  Dirk Merkel,et al.  Docker: lightweight Linux containers for consistent development and deployment , 2014 .