Enabling Docker Containers for High-Performance and Many-Task Computing

Docker is the most popular and user friendlyplatform for running and managing Linux containers. This isproven by the fact that vast majority of containerized tools arepackaged as Docker images. A demanding functionality is toenable running Docker containers inside HPC job scripts forresearchers to make use of the flexibility offered by containersin their real-life computational and data intensive jobs. The maintwo questions before implementing such functionality are: how tosecurely run Docker containers within cluster jobs? and how tolimit the resource usage of a Docker job to the borders defined bythe HPC queuing system? This paper presents Socker, a securewrapper for running Docker containers on Slurm and similarqueuing systems. Socker enforces the execution of containerswithin Slurm jobs as the submitting user instead of root, as wellas enforcing the inclusion of containers in the cgroups assignedby the queuing system to the parent jobs. Different from otherDocker supported containers-for-hpc platform, socker uses theunderlaying Docker engine instead of replacing it. To eveluatesocker, it has been tested for running MPI Docker jobs on Slurm. It has been also tested for Many-task computing (MTC) on interconnectedclusters. Socker has proven to be secure, as well asintroducing no additional overhead to the one introduced alreadyby the Docker engine.