ResearchCompendia.org: Cyberinfrastructure for Reproducibility and Collaboration in Computational Science

We outline three goals to consider in building cyberinfrastructure to support scientific research and dissemination, and present our demonstration project ResearchCompendia. We posit that cyberinfrastructure should reinforce scientific norms, such as transparency and reproducibility, while embedding and encouraging best practices in scientific research, such as citation. Finally, we believe cyberinfrastucture should consider the entire soup-to-nuts discovery pipeline, even if focusing only on a subset of the workflow. In this article, we develop these ideas in the context of the ResearchCompendia project. ResearchCompendia is designed to facilitate reproducibility in computational science by persistently linking data and code that generated published findings to the article, and executing the code in the cloud to validate or certify those findings. We conclude with a discussion of the future vision of cyberinfrastructure and ResearchCompendia in support of science.

[1]  Carole A. Goble,et al.  The design and realisation of the myExperiment Virtual Research Environment for social sharing of workflows , 2009, Future Gener. Comput. Syst..

[2]  Stability , 1973 .

[3]  Patrick J. Roache,et al.  Perspective: Validation—What Does It Mean? , 2009 .

[4]  Victoria Stodden,et al.  Enabling reproducible research: Licensing for scientific innovation , 2009 .

[5]  D. Madigan,et al.  A Systematic Statistical Approach to Evaluating Evidence from Observational Studies , 2014 .

[6]  Jennifer M. Urban,et al.  Shining Light into Black Boxes , 2012, Science.

[7]  Michael McLennan,et al.  HUBzero: A Platform for Dissemination and Collaboration in Computational Science and Engineering , 2010, Computing in Science & Engineering.

[8]  Robert Gentleman,et al.  Statistical Analyses and Reproducible Research , 2007 .

[9]  Victoria Stodden,et al.  The Legal Framework for Reproducible Scientific Research: Licensing and Copyright , 2009, Computing in Science & Engineering.

[10]  Norman Kaplan,et al.  The Sociology of Science: Theoretical and Empirical Investigations , 1974 .

[11]  P. N. Edwards,et al.  Knowledge Infrastructures: Intellectual Frameworks and Research Challenges , 2013 .

[12]  Brigid Wilson,et al.  Implementing Reproducible Research , 2014 .

[13]  Victoria Stodden,et al.  RunMyCode.org: A novel dissemination and collaboration platform for executing published computational results , 2012, 2012 IEEE 8th International Conference on E-Science.

[14]  Jonathan M. Borwein,et al.  Opinion: set the default to "open" , 2013 .

[15]  Victoria Stodden,et al.  The Scientific Method in Practice: Reproducibility in the Computational Sciences , 2010 .

[16]  V. Stodden,et al.  Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals , 2013, PloS one.

[17]  R. Merton.  The Normative Structure of Science , 1973 .

[18]  Ian M. Mitchell,et al.  Reproducible research for scientific computing: Tools and strategies for changing the culture , 2012, Computing in Science & Engineering.

[19]  Arian Maleki,et al.  Reproducible Research in Computational Harmonic Analysis , 2009, Computing in Science & Engineering.