A Continuous Integration-Based Framework for Software Management

Automated software-installation tools such as EasyBuild and Spack have enabled HPC support staff to more quickly deploy software environments for HPC. However, to provide reliable user-facing software environments, any change made to the environments should be recorded and tested. In this paper, we will describe a continuous integration-based framework for software management in use at the Ohio Supercomputer Center (OSC). In this framework, we rely on homegrown tools for automated software build and installation and the ReFrame framework to setup the testing system which can be triggered on commit to a local Gitlab repository. We will document the workflow and explain how we can deploy the software environment quickly and confidently using these tools.

[1]  A regression framework for checking the health of large HPC systems , 2017 .

[2]  Bronis R. de Supinski,et al.  The Spack package manager: bringing order to HPC software chaos , 2015, SC15: International Conference for High Performance Computing, Networking, Storage and Analysis.

[3]  Thomas Hauser,et al.  Continuous Integration and Delivery for HPC: Using Singularity and Jenkins , 2018, PEARC.

[4]  Tommy Minyard,et al.  Best practices for the deployment and management of production HPC clusters , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[5]  Jack J. Dongarra,et al.  Collecting Performance Data with PAPI-C , 2009, Parallel Tools Workshop.

[6]  Markus Geimer,et al.  Modern Scientific Software Management Using EasyBuild and Lmod , 2014, 2014 First International Workshop on HPC User Support Tools.

[7]  Amiya K. Maji,et al.  Testpilot: A Flexible Framework for User-centric Testing of HPC Clusters , 2017 .