Proposal of Container-Based HPC Structures and Performance Analysis

High-performance computing (HPC) provides to researchers a powerful ability to resolve problems with intensive computations, such as those in the math and medical fields. When an HPC platform is provided as a service, users may suffer from unexpected obstacles in developing and running applications due to restricted development environments and dependencies. In this context, operating system level virtualization can be a solution for HPC service to ensure lightweight virtualization and consistency in Dev-Ops environments. Therefore, this paper proposes three types of typical HPC structure for container environments built with HPC container and Docker. The three structures focus on smooth integration with existing HPC job framework, message passing interface (MPI). Lastly, the performance of the structures is analyzed with High Performance Linpack benchmark from the aspect of performance degradation in network communications under Docker.

[1]  Doina Bein,et al.  Distributed MPI cluster with Docker Swarm mode , 2017, 2017 IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC).

[3]  Maximilien de Bayser,et al.  Integrating MPI with Docker for HPC , 2017, 2017 IEEE International Conference on Cloud Engineering (IC2E).

[4]  Jun-Ho Huh,et al.  Design and test bed experiments of server operation system using virtualization technology , 2016, Human-centric Computing and Information Sciences.

[5]  Vanessa Sochat,et al.  Singularity: Scientific containers for mobility of compute , 2017, PloS one.

[6]  Violeta Holmes,et al.  Orchestrating Docker Containers in the HPC Environment , 2015, ISC.

[7]  Lucas Benedicic,et al.  Portable, high-performance containers for HPC , 2017, ArXiv.