Integrating MPI with Docker for HPC

Container technology has the potential to considerably simplify the management of the software stack of High Performance Computing (HPC) clusters. However, poor integration with established HPC technologies is still preventing users and administrators to reap the benefits of containers. Message Passing Interface (MPI) is a pervasive technology used to run scientific software, often written in Fortran and C/C++, that presents challenges for effective integration with containers. This work shows how an existing MPI implementation can be extended to improve this integration.

[1]  Weicheng Huang,et al.  Building a Virtual HPC Cluster with Auto Scaling by the Docker , 2015, ArXiv.

[2]  Rajkumar Buyya,et al.  Cloud Computing Principles and Paradigms , 2011 .

[3]  Christian S. Collberg,et al.  Repeatability in computer systems research , 2016, Commun. ACM.

[4]  Eric A. Brewer,et al.  Borg, Omega, and Kubernetes , 2016, ACM Queue.

[5]  Stephen Childs,et al.  GridBuilder: A Tool for Creating Virtual Grid Testbeds , 2006, 2006 Second IEEE International Conference on e-Science and Grid Computing (e-Science'06).

[6]  Chris Morris,et al.  Developing Software for a Scientific Community: Some Challenges and Solutions , 2011 .

[7]  Carl Boettiger,et al.  An introduction to Docker for reproducible research , 2014, OPSR.

[8]  Cees T. A. M. de Laat,et al.  AMOS: Using the Cloud for On-Demand Execution of e-Science Applications , 2010, 2010 IEEE Sixth International Conference on e-Science.

[9]  Eric Eide,et al.  Toward Replayable Research in Networking and Systems , 2010 .

[10]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[11]  Joel T Dudley,et al.  In silico research in the era of cloud computing , 2010, Nature Biotechnology.

[12]  William Johnson,et al.  High-performance computing without commitment: SC2IT: A cloud computing interface that makes computational science available to non-specialists , 2012, 2012 IEEE 8th International Conference on E-Science.

[13]  Shantenu Jha,et al.  An Autonomic Approach to Integrated HPC Grid and Cloud Usage , 2009, 2009 Fifth IEEE International Conference on e-Science.

[14]  Nick Barnes.  Publish your computer code: it is good enough , 2010, Nature.

[15]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[16]  Erich Strohmaier,et al.  TOP500 supercomputer , 2006, SC.