Special Issue on Reproducible Research for Biomedical Informatics

Reproducible research is of significant concern for researchers, clinical practitioners, and patients globally [1,2]. To further the scientific method and improve health outcomes, practices and processes for true reproducibility extend beyond the methods section of a journal article and into the references as well as the availability of a specific time-stamped repository and databases: query code, research protocols, software code and versioning, datasets, metadata, and more [3]. Given the protected nature of much biomedical research, reproducible research in this domain is complicated, yet still necessary to treat and solve real-world health issues. Though the success of clinical care and trials is heavily dependent upon the validity and reliability of previous research [4], the practices and methods that researchers use to improve the reproducibility of their research is not well documented. The inability to replicate and reproduce research published in prestigious journals is an area of growing significance [5–8]. Given the potential translation of this research to patient bedside practices, and the need to ensure the most effective use of research funds, high-quality, thorough research methods, workflows and documentation are a necessity for all empirical, computational and analytical research [9]. Much has been published in specific domains about theoretical practices for ensuring research validity. Landis [10] outlines a core set of reporting standards of rigorous study design for preclinical research involving animal studies that cover topics such as randomization, blinding, sample-size estimation, and data handling. Similarly, in the area of psychology, workflows for testing the reproducibility of research are coming to fruition [11]. Yet, the practical methods actually used within the biomedical informatics domain are not well documented or have not historically been positioned within the reproducible research framework. For example, metadata are a strong component of facilitating reproducible research, and the Journal of Biomedical Informatics has previously published on metadata, as a mechanism to facilitate interoperability [12]. Additionally, documentation of information has been covered in various articles within the Journal of Biomedical Informatics [13]. Fields also either within the scope of biomedical informatics but with other primary journals (e.g., imaging informatics [14]), as well as those with overlapping collaborations within biomedical informatics (e.g., biostatistics [15] and computational science [16]) have already published on this topic. Finally, reproducibility hinges not only on the availability of the documentation, analyses, and data, but on the ongoing accessibility and viability of the files and documents, enhanced through a process of curation. Biocuration enables information discovery and retrieval, maintains quality, adds value, and provides for re-use over time through activities including authentication, archiving, management, preservation, and representation [17]. Much published research has focused on the workflows and case studies for conducting text mining to extract data from published literature as a process of biocuration [18–20], yet this is a limited view of curation, and does not support quality information provenance. Within this context of reproducible research for biomedical informatics, we encourage you to submit articles about methods to support, improve, validate, or assess the reproducibility of biomedical informatics research. Topics of interest for submission to this special issue include (but are not limited to):