Everything Matters: The ReproNim Perspective on Reproducible Neuroimaging

There has been a recent major upsurge in the concerns about reproducibility in many areas of science. Within the neuroimaging domain, one approach is to promote reproducibility is to target the re-executability of the publication. The information supporting such re-executability can enable the detailed examination of how an initial finding generalizes across changes in the processing approach, and sampled population, in a controlled scientific fashion. ReproNim: A Center for Reproducible Neuroimaging Computation is a recently funded initiative that seeks to facilitate the “last mile” implementations of core re-executability tools in order to reduce the accessibility barrier and increase adoption of standards and best practices at the neuroimaging research laboratory level. In this report, we summarize the overall approach and tools we have developed in this domain.

[1]  C. Drummond Replicability is not Reproducibility:Nor is it Good Science , 2009 .

[2]  DN Kennedy ReproNim Simple Workflow test dataset , 2017 .

[3]  Bharat B. Biswal,et al.  Making data sharing work: The FCP/INDI experience , 2013, NeuroImage.

[4]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[5]  David B. Keator,et al.  Towards structured sharing of raw and derived neuroimaging data across existing resources , 2012, NeuroImage.

[6]  N. Lazar,et al.  The ASA Statement on p-Values: Context, Process, and Purpose , 2016 .

[7]  Satrajit S. Ghosh,et al.  Nipype: A Flexible, Lightweight and Extensible Neuroimaging Data Processing Framework in Python , 2011, Front. Neuroinform..

[8]  Maryann E. Martone,et al.  FORCE11: Building the Future for Research Communications and e-Scholarship , 2015 .

[9]  Arthur W Toga,et al.  The LONI Pipeline Processing Environment , 2003, NeuroImage.

[10]  Satrajit S. Ghosh,et al.  The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments , 2016, Scientific Data.

[11]  David N. Kennedy,et al.  The NITRC image repository , 2016, NeuroImage.

[12]  Abraham Z. Snyder,et al.  Human Connectome Project informatics: Quality control, database services, and data visualization , 2013, NeuroImage.

[13]  Wenli Ma,et al.  The human hippocampus is not sexually-dimorphic: Meta-analysis of structural MRI volumes , 2016, NeuroImage.

[14]  Tal Kenet,et al.  The Pediatric Imaging, Neurocognition, and Genetics (PING) Data Repository , 2016, NeuroImage.

[15]  Shoaib Sufi,et al.  Toward standard practices for sharing computer code and programs in neuroscience , 2016, Nature Neuroscience.

[16]  Chue Hong Neil Open Software for (Open) Science , 2015 .

[17]  Arno Klein,et al.  101 Labeled Brain Images and a Consistent Human Cortical Labeling Protocol , 2012, Front. Neurosci..

[18]  David N. Kennedy,et al.  Tools Matter: Comparison of Two Surface Analysis Tools Applied to the ABIDE Dataset , 2017 .

[19]  Ze Wang A very simple , re-executable neuroimaging publication , 2018 .

[20]  Kevin A. Archie,et al.  XNAT Central: Open sourcing imaging research data , 2016, NeuroImage.

[21]  John P A Ioannidis,et al.  Excess significance bias in the literature on brain volume abnormalities. , 2011, Archives of general psychiatry.

[22]  A. Dale,et al.  Whole Brain Segmentation Automated Labeling of Neuroanatomical Structures in the Human Brain , 2002, Neuron.

[23]  Alan C. Evans,et al.  The NIH MRI study of normal brain development , 2006, NeuroImage.

[24]  John P. A. Ioannidis,et al.  What does research reproducibility mean? , 2016, Science Translational Medicine.

[25]  Erik Schultes,et al.  The FAIR Guiding Principles for scientific data management and stewardship , 2016, Scientific Data.

[26]  Brian B. Avants,et al.  An Open Source Multivariate Framework for n-Tissue Segmentation with Evaluation on Public Data , 2011, Neuroinformatics.

[27]  Arno Klein,et al.  Large-scale evaluation of ANTs and FreeSurfer cortical thickness measurements , 2014, NeuroImage.

[28]  Leif D. Nelson,et al.  P-Curve: A Key to the File Drawer , 2013, Journal of experimental psychology. General.

[29]  R. Peng Reproducible Research in Computational Science , 2011, Science.

[30]  Daniel P. Kennedy,et al.  The Autism Brain Imaging Data Exchange: Towards Large-Scale Evaluation of the Intrinsic Brain Architecture in Autism , 2013, Molecular Psychiatry.

[31]  Ruth E. Duerr,et al.  Achieving human and machine accessibility of cited data in scholarly publications , 2015, PeerJ Comput. Sci..

[32]  Joanne Godley,et al.  Down the Rabbit Hole , 2013, Narrative inquiry in bioethics.

[33]  Tristan Glatard,et al.  Reproducibility of neuroimaging analyses across operating systems , 2015, Front. Neuroinform..

[34]  Thomas E. Nichols,et al.  Best practices in data analysis and sharing in neuroimaging using MRI , 2017, Nature Neuroscience.

[35]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.