Containers for Portable, Productive, and Performant Scientific Computing

Containers are an emerging technology that holds promise for improving productivity and code portability in scientific computing. The authors examine Linux container technology for the distribution of a nontrivial scientific computing software stack and its execution on a spectrum of platforms from laptop computers through high-performance computing systems. For Python code run on large parallel computers, the runtime is reduced inside a container due to faster library imports. The software distribution approach and data that the authors present will help developers and users decide on whether container technology is appropriate for them. The article also provides guidance for vendors of HPC systems that rely on proprietary libraries for performance on what they can do to make containers work seamlessly and without performance penalty.

[1]  S. HaleJack,et al.  Containers for portable, productive and performant scientific computing; supporting data and code. , 2016 .

[2]  Andy Brown,et al.  ABC: Using Object Tracking to Automate Behavioural Coding , 2016, CHI Extended Abstracts.

[3]  Dirk Merkel,et al.  Docker: lightweight Linux containers for consistent development and deployment , 2014 .

[4]  Introduction to Container Security , .

[5]  Ramakrishnan Rajamony,et al.  An updated performance comparison of virtual machines and Linux containers , 2015, 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[6]  Donald Bahls Evaluating Shifter for HPC Applications , 2016 .

[7]  D. Jacobsen,et al.  Contain This, Unleashing Docker for HPC , 2015 .

[8]  Pablo Prieto,et al.  The impact of Docker containers on the performance of genomic pipelines , 2015, PeerJ.

[9]  Carl Boettiger,et al.  An introduction to Docker for reproducible research , 2014, OPSR.

[10]  John Shalf,et al.  HPGMG 1.0: A Benchmark for Ranking High Performance Computing Systems , 2014 .

[11]  Brian E. Granger,et al.  IPython: A System for Interactive Scientific Computing , 2007, Computing in Science & Engineering.

[12]  Oh-Young Kwon,et al.  Performance Comparison Analysis of Linux Container and Virtual Machine for Building Cloud , 2014 .

[13]  Andreas Wilke,et al.  Skyport - Container-Based Execution Environment Management for Multi-cloud Scientific Workflows , 2014, 2014 5th International Workshop on Data-Intensive Computing in the Clouds.

[14]  Anders Logg,et al.  Automated Solution of Differential Equations by the Finite Element Method: The FEniCS Book , 2012 .

[15]  Paul Lu,et al.  Pragmatics of Virtual Machines for High-Performance Computing : A Quantitative Study of Basic Overheads Cam Macdonell and , 2007 .