Report on the Second Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE2)

This technical report records and discusses the Second Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE2). The report includes a description of the alternative, experimental submission and review process, two workshop keynote presentations, a series of lightning talks, a discussion on sustainability, and five discussions from the topic areas of exploring sustainability; software development experiences; credit & incentives; reproducibility & reuse & sharing; and code testing & code review. For each topic, the report includes a list of tangible actions that were proposed and that would lead to potential change. The workshop recognized that reliance on scientific software is pervasive in all areas of world-leading research today. The workshop participants then proceeded to explore different perspectives on the concept of sustainability. Key enablers and barriers of sustainable scientific software were identified from their experiences. In addition, recommendations with new requirements such as software credit files and software prize frameworks were outlined for improving practices in sustainable software engineering. There was also broad consensus that formal training in software development or engineering was rare among the practitioners. Significant strides need to be made in building a sense of community via training in software and technical practices, on increasing their size and scope, and on better integrating them directly into graduate education programs. Finally, journals can define and publish policies to improve reproducibility, whereas reviewers can insist that authors provide sufficient information and access to data and software to allow them reproduce the results in the paper. Hence a list of criteria is compiled for journals to provide to reviewers so as to make it easier to review software submitted for publication as a "Software Paper."

[1]  Ivica Crnkovic,et al.  Framing sustainability as a property of software quality , 2015, Commun. ACM.

[2]  David R. Barstow,et al.  An automatic programming system to support an experimental science , 1982, ICSE '82.

[3]  Luis V. Gómez-Ontiveros.  Open Researcher and Contributor ID (ORCID) , 2017 .

[4]  Brian Matthews,et al.  Guidelines for persistently identifying software using DataCite , 2015 .

[5]  James S. Spencer,et al.  The Highly Accurate N-DEterminant (HANDE) quantum Monte Carlo project: Open-source stochastic diagonalisation for quantum chemistry , 2014, ArXiv.

[6]  Konrad Hinsen,et al.  The Hierarchical Data Format (HDF): A Foundation for Sustainable Data and Software , 2014 .

[7]  Scott Chamberlain,et al.  Building Software, Building Community: Lessons from the rOpenSci Project , 2014 .

[8]  Chamberlain Ryan,et al.  Using Docker to Support Reproducible Research , 2014 .

[9]  Marcus D. Hanwell,et al.  Sustainable Software Ecosystems: Software Engineers, Domain Scientists, and Engineers Collaborating for Science , 2014, ArXiv.

[10]  Howison James.  Retract bit-rotten publications: Aligning incentives for sustaining scientific software , 2014 .

[11]  Ray Idaszak,et al.  ISEES-­‐WSSI Lessons for Sustainable Science Software from an Early Career Training Institute on Open Science Synthesis , 2014 .

[12]  Markus Blatt,et al.  DUNE as an Example of Sustainable Open Source Scientific Software Development , 2013, ArXiv.

[13]  Colin C. Venters,et al.  The Nebuchadnezzar Effect: Dreaming of Sustainable Software through Sustainable Software Architectures , 2014 .

[14]  Justin Shi.  Seeking the Principles of Sustainable Software Engineering , 2014 .

[15]  Jory Schossau,et al.  Which Sustainable Software Practices Do Scientists Find Most Useful? , 2014, ArXiv.

[16]  Daniel S. Katz,et al.  Report on the Third Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3) , 2016, ArXiv.

[17]  Alice Allen,et al.  Looking before leaping: Creating a software registry , 2014, ArXiv.

[18]  Andrew Siegel,et al.  Extensible component-based architecture for FLASH, a massively parallel, multiphysics simulation code , 2009, Parallel Comput..

[19]  Marru Suresh,et al.  WSSSPE2: Patching It Up, Pulling It Forward , 2014 .

[20]  Diane Kelly,et al.  Testing for trustworthiness in scientific software , 2009, 2009 ICSE Workshop on Software Engineering for Computational Science and Engineering.

[21]  Joel Spolsky.  The Joel Test: 12 Steps to Better Code , 2004 .

[22]  Diane Kelly,et al.  Mutation Sensitivity Testing , 2009, Computing in Science & Engineering.

[23]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.