Replication and generalization in applied neuroimaging

There is much interest in translating neuroimaging findings into meaningful clinical diagnostics. The goal of scientific discoveries differs from clinical diagnostics. Scientific discoveries must replicate under a specific set of conditions; to translate to the clinic we must show that findings using purpose-built scientific instruments will be observable in clinical populations and instruments. Here we describe and evaluate data and computational methods designed to translate a scientific observation to a clinical setting. Using diffusion weighted imaging (DWI), Wahl et al. (2010) observed that across subjects the mean fractional anisotropy (FA) of homologous pairs of tracts is highly correlated. We hypothesize that this is a fundamental biological trait that should be present in most healthy participants, and deviations from this assessment may be a useful diagnostic metric. Using this metric as an illustration of our methods, we analyzed six pairs of homologous white matter tracts in nine different DWI datasets with 44 subjects each. Considering the original FA measurement as a baseline, we show that the new metric is between 2 and 4 times more precise when used in a clinical context. Our framework to translate research findings into clinical practice can be applied, in principle, to other neuroimaging results.

[1]  T. Ellis,et al.  A New Initiative. , 2017, Water environment research.

[2]  Essa Yacoub,et al.  The Lifespan Human Connectome Project in Aging: An overview , 2019, NeuroImage.

[3]  Stamatios N. Sotiropoulos,et al.  Non-parametric representation and prediction of single- and multi-shell diffusion-weighted MRI data using Gaussian processes , 2015, NeuroImage.

[4]  R. Peng Reproducible Research in Computational Science , 2011, Science.

[5]  Steen Moeller,et al.  The Human Connectome Project's neuroimaging approach , 2016, Nature Neuroscience.

[6]  Stephen Senn,et al.  Statistical pitfalls of personalized medicine , 2018, Nature.

[7]  Timothy O. Laumann,et al.  Informatics and Data Mining Tools and Strategies for the Human Connectome Project , 2011, Front. Neuroinform..

[8]  Joshua Ng,et al.  Microstructural correlations of white matter tracts in the human brain , 2010, NeuroImage.

[9]  Paul M. Thompson,et al.  A resting state fMRI analysis pipeline for pooling inference across diverse cohorts: an ENIGMA rs-fMRI protocol , 2018, Brain Imaging and Behavior.

[10]  Elizabeth Tipton,et al.  How Generalizable Is Your Experiment? An Index for Comparing Experimental Samples and Populations , 2014 .

[11]  Robert I. Reid,et al.  THE ADNI3 DIFFUSION MRI PROTOCOL: BASIC + ADVANCED , 2017, Alzheimer's & Dementia.

[12]  Jan Sijbers,et al.  Denoising of diffusion MRI using random matrix theory , 2016, NeuroImage.

[13]  Mark Jenkinson,et al.  The minimal preprocessing pipelines for the Human Connectome Project , 2013, NeuroImage.

[14]  Derek K. Jones,et al.  Symmetries in human brain language pathways correlate with verbal recall , 2007, Proceedings of the National Academy of Sciences.

[15]  Stefan Skare,et al.  How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging , 2003, NeuroImage.

[16]  Jerry L. Prince,et al.  Effects of diffusion weighting schemes on the reproducibility of DTI-derived fractional anisotropy, mean diffusivity, and principal eigenvector measurements at 1.5T , 2007, NeuroImage.

[17]  Patrick B. Ryan,et al.  Simulation-based Evaluation of the Generalizability Index for Study Traits , 2015, AMIA.

[18]  Thomas E. Nichols,et al.  Extending the Human Connectome Project across ages: Imaging protocols for the Lifespan Development and Aging projects , 2018, NeuroImage.

[19]  Paul M. Thompson,et al.  Diffusion MRI Indices and Their Relation to Cognitive Impairment in Brain Aging: The Updated Multi-protocol Approach in ADNI3 , 2018, bioRxiv.

[20]  F. Goodsaid,et al.  Translational Medicine and the Value of Biomarker Qualification , 2010, Science Translational Medicine.

[21]  Stamatios N. Sotiropoulos,et al.  An integrated approach to correction for off-resonance effects and subject movement in diffusion MR imaging , 2016, NeuroImage.

[22]  Max A. Viergever,et al.  Reducing variability in along-tract analysis with diffusion profile realignment⋆ , 2019, bioRxiv.

[23]  Max A. Viergever,et al.  Reducing variability in along-tract analysis with diffusion profile realignment , 2019, NeuroImage.

[24]  Anton Nekrutenko,et al.  Ten Simple Rules for Reproducible Computational Research , 2013, PLoS Comput. Biol..

[25]  B. Wandell,et al.  Lifespan maturation and degeneration of human brain white matter , 2014, Nature Communications.

[26]  Richard J. Shavelson,et al.  Generalizability Theory: A Primer , 1991 .

[27]  S. A. Wijtenburg,et al.  Reproducibility of tract‐based white matter microstructural measures using the ENIGMA‐DTI protocol , 2017, Brain and behavior.

[28]  Michael L. Perry,et al.  Reproducible Tract Profiles (RTP): from diffusion MRI acquisition to publication , 2019 .

[29]  Victoria Stodden,et al.  Implementing Reproducible Research , 2018 .

[30]  Marco Rovaris,et al.  Diffusion tensor MR imaging. , 2009, Neuroimaging clinics of North America.

[31]  Joshua Carp,et al.  On the Plurality of (Methodological) Worlds: Estimating the Analytic Flexibility of fMRI Experiments , 2012, Front. Neurosci..

[32]  Jelle Veraart,et al.  Diffusion MRI noise mapping using random matrix theory , 2016, Magnetic resonance in medicine.

[33]  C. Lebel,et al.  Lateralization of the arcuate fasciculus from childhood to adulthood and its relation to cognitive abilities in children , 2009, Human brain mapping.

[34]  Susumu Mori,et al.  Effects of b-value and echo time on magnetic resonance diffusion tensor imaging-derived parameters at 1.5 t: A voxel-wise study , 2013 .

[35]  Brian B. Avants,et al.  N4ITK: Improved N3 Bias Correction , 2010, IEEE Transactions on Medical Imaging.

[36]  M. Baker 1,500 scientists lift the lid on reproducibility , 2016, Nature.

[37]  T. Yarkoni,et al.  Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning , 2017, Perspectives on psychological science : a journal of the Association for Psychological Science.

[38]  D. Schnyer,et al.  Toward Precision and Reproducibility of Diffusion Tensor Imaging: A Multicenter Diffusion Phantom and Traveling Volunteer Study , 2016, American Journal of Neuroradiology.

[39]  Lex Nederbragt,et al.  Good enough practices in scientific computing , 2016, PLoS Comput. Biol..

[40]  Brian A. Wandell,et al.  Anatomical Properties of the Arcuate Fasciculus Predict Phonological and Reading Skills in Children , 2011, Journal of Cognitive Neuroscience.

[41]  J. Ioannidis,et al.  Meta-analysis methods for genome-wide association studies and beyond , 2013, Nature Reviews Genetics.

[42]  Richard Mayeux,et al.  Biomarkers: Potential uses and limitations , 2004 .

[43]  Brian A. Wandell,et al.  Ensemble Tractography , 2016, PLoS Comput. Biol..

[44]  B. Wandell,et al.  Tract Profiles of White Matter Properties: Automating Fiber-Tract Quantification , 2012, PloS one.

[45]  S. Naylor,et al.  Biomarkers: current perspectives and future prospects , 2003, Expert review of molecular diagnostics.

[46]  Satrajit S. Ghosh,et al.  Everything Matters: The ReproNim Perspective on Reproducible Neuroimaging , 2018, Front. Neuroinform..

[47]  Essa Yacoub,et al.  The Lifespan Human Connectome Project in Development: A large-scale study of brain connectivity development in 5–21 year olds , 2018, NeuroImage.

[48]  Bibek Dhital,et al.  Gibbs‐ringing artifact removal based on local subvoxel‐shifts , 2015, Magnetic resonance in medicine.

[49]  Thomas E. Nichols,et al.  Scanning the horizon: towards transparent and reproducible neuroimaging research , 2016, Nature Reviews Neuroscience.

[50]  A. Malhotra,et al.  Integration of routine QA data into mega‐analysis may improve quality and sensitivity of multisite diffusion tensor imaging studies , 2018, Human Brain Mapping.

[51]  Hans Ekkehard Plesser,et al.  Reproducibility vs. Replicability: A Brief History of a Confused Terminology , 2018, Front. Neuroinform..

[52]  Jerry L Prince,et al.  Effects of signal‐to‐noise ratio on the accuracy and reproducibility of diffusion tensor imaging–derived fractional anisotropy, mean diffusivity, and principal eigenvector measurements at 1.5T , 2007, Journal of magnetic resonance imaging : JMRI.

[53]  F. Collins,et al.  A new initiative on precision medicine. , 2015, The New England journal of medicine.

[54]  Pratik Mukherjee,et al.  Magnetic resonance imaging improves 3‐month outcome prediction in mild traumatic brain injury , 2012, Annals of neurology.

[55]  Derek K. Jones,et al.  “Squashing peanuts and smashing pumpkins”: How noise distorts diffusion‐weighted MR data , 2004, Magnetic resonance in medicine.

[56]  Prasad Patil,et al.  A statistical definition for reproducibility and replicability , 2016, bioRxiv.

[57]  R. Henry,et al.  Diffusion Tensor MR Imaging and Fiber Tractography: Theoretic Underpinnings , 2008, American Journal of Neuroradiology.

[58]  L. Simpson Report on Certain Enteric Fever Inoculation Statistics , 1904, British medical journal.

[59]  Carlo Pierpaoli,et al.  Analysis of the effects of noise, DWI sampling, and value of assumed parameters in diffusion MRI models , 2017, Magnetic resonance in medicine.

[60]  R. Henry,et al.  Diffusion Tensor MR Imaging and Fiber Tractography: Technical Considerations , 2008, American Journal of Neuroradiology.

[61]  John P. A. Ioannidis,et al.  What does research reproducibility mean? , 2016, Science Translational Medicine.

[62]  Ariel Rokem,et al.  Evaluating the Accuracy of Diffusion MRI Models in White Matter , 2015, PloS one.

[63]  Nigel O'Brian,et al.  Generalizability Theory I , 2003 .