Multivariate consistency of resting-state fMRI connectivity maps acquired on a single individual over 2.5 years, 13 sites and 3 vendors

Studies using resting-state functional magnetic resonance imaging (rsfMRI) are increasingly collecting data at multiple sites in order to speed up recruitment or increase sample size. The main objective of this study was to assess the long-term consistency of rsfMRI connectivity maps derived at multiple sites and vendors using the Canadian Dementia Imaging Protocol (CDIP, www.cdip-pcid.ca). Nine to ten minutes of functional BOLD images were acquired from an adult cognitively healthy volunteer scanned repeatedly at 13 Canadian sites on three scanner makes (General Electric, Philips and Siemens) over the course of 2.5 years. The consistency (spatial Pearson’s correlation) of rsfMRI connectivity maps for seven canonical networks ranged from about 0.4-0.8 (intra-site) to 0.3-0.8 (inter-vendor), with a negligible effect of time. We noted systematic differences in data quality across vendors, which may also explain some of these results. We also pooled the long-term longitudinal data with a single-site, short-term (1 month) data sample acquired on 26 subjects (10 scans per subject), called HNU1. Using randomly selected pairs of scans from each subject, we quantified the ability of a data-driven unsupervised cluster analysis to match two scans of the same subjects. In this “fingerprinting” experiment, we found that scans from the Canadian subject (Csub) could be matched with high accuracy intra-site (>95% for some networks), but that the accuracy decreased substantially for scans drawn from different sites and vendors, while still remaining in the range of accuracies observed in HNU1. Overall, our results demonstrate good multivariate stability of rsfMRI measures over several years, but substantial impact of scanning site and vendors. How detrimental these effects are will depend on the application, yet improving methods for harmonizing multisite analysis is an important area for future work. HIGHLIGHTS Consistency of rsfMRI connectivity over 2.5 years, 13 sites and 3 scanner vendors Time elapsed between scans had negligible effect on consistency Consistency decreased due to site and vendor differences Accuracy of connectivity fingerprints decreased due to site and vendor differences Graphical Abstract

[1]  Skipper Seabold,et al.  Statsmodels: Econometric and Statistical Modeling with Python , 2010, SciPy.

[2]  Daniel P. Kennedy,et al.  Enhancing studies of the connectome in autism using the autism brain imaging data exchange II , 2017, Scientific Data.

[3]  Xi-Nian Zuo,et al.  Toward systems neuroscience in mild cognitive impairment and Alzheimer's disease: A meta‐analysis of 75 fMRI studies , 2015, Human brain mapping.

[4]  Russell Greiner,et al.  ADHD-200 Global Competition: diagnosing ADHD using personal characteristic data can outperform resting state fMRI measurements , 2012, Front. Syst. Neurosci..

[5]  M. Chun,et al.  Functional connectome fingerprinting: Identifying individuals based on patterns of brain connectivity , 2015, Nature Neuroscience.

[6]  Emily S. Finn,et al.  Individual variation in functional brain connectivity: implications for personalized approaches to psychiatric disease , 2016, Dialogues in clinical neuroscience.

[7]  Emiliano Macaluso,et al.  Images-based suppression of unwanted global signals in resting-state functional connectivity studies. , 2009, Magnetic resonance imaging.

[8]  Jang-Yeon Park,et al.  Inter-vender and test-retest reliabilities of resting-state functional magnetic resonance imaging: Implications for multi-center imaging studies. , 2017, Magnetic resonance imaging.

[9]  D. Cicchetti,et al.  Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. , 1981, American journal of mental deficiency.

[10]  Pierre Bellec,et al.  MIST: A multi-resolution parcellation of functional brain networks , 2017, MNI Open Research.

[11]  Moors Pieter,et al.  Test-retest reliability. , 2014 .

[12]  Emilio Merlo Pich,et al.  Statistical power and prediction accuracy in multisite resting-state fMRI connectivity , 2017, NeuroImage.

[13]  Timothy O. Laumann,et al.  Functional Brain Networks Are Dominated by Stable Group and Individual Factors, Not Cognitive or Daily Variation , 2018, Neuron.

[14]  P. Jezzard,et al.  Correction for geometric distortion in echo planar images from B0 field variations , 1995, Magnetic resonance in medicine.

[15]  P. Matthews,et al.  Clinical Concepts Emerging from fMRI Functional Connectomics , 2016, Neuron.

[16]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[17]  Ludovico Minati,et al.  Longitudinal reproducibility of default-mode network connectivity in healthy elderly participants: A multicentric resting-state fMRI study , 2016, NeuroImage.

[18]  Pierre Bellec,et al.  Resting-state network dysfunction in Alzheimer's disease: A systematic review and meta-analysis , 2017, Alzheimer's & dementia.

[19]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[20]  Jacob Cohen,et al.  The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability , 1973 .

[21]  Travis E. Oliphant,et al.  Guide to NumPy , 2015 .

[22]  Wei Cheng,et al.  Voxel-based, brain-wide association study of aberrant functional connectivity in schizophrenia implicates thalamocortical circuitry , 2015, npj Schizophrenia.

[23]  Dustin Scheinost,et al.  Influences on the Test–Retest Reliability of Functional Connectivity MRI and its Relationship with Behavioral Utility , 2017, Cerebral cortex.

[24]  Wes McKinney,et al.  Data Structures for Statistical Computing in Python , 2010, SciPy.

[25]  Dimitris Samaras,et al.  Deriving reproducible biomarkers from multi-site resting-state data: An Autism-based example , 2016, NeuroImage.

[26]  J. Trojanowski,et al.  Heterogeneity of neuroanatomical patterns in prodromal Alzheimer’s disease: links to cognition, progression and biomarkers , 2016, Brain : a journal of neurology.

[27]  Allan L. Reiss,et al.  Compensatory Hyperconnectivity in Developing Brains of Young Children With Type 1 Diabetes , 2016, Diabetes.

[28]  Lars T. Westlye,et al.  Consistent Functional Connectivity Alterations in Schizophrenia Spectrum Disorder: A Multisite Study , 2017, Schizophrenia bulletin.

[29]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[30]  Lee Friedman,et al.  Report on a multicenter fMRI quality assurance protocol , 2006, Journal of magnetic resonance imaging : JMRI.

[31]  Pierre Orban,et al.  Multisite generalizability of schizophrenia diagnosis classification based on functional brain connectivity , 2018, Schizophrenia Research.

[32]  Evan M. Gordon,et al.  Functional System and Areal Organization of a Highly Sampled Individual Human Brain , 2015, Neuron.

[33]  Yong Fan,et al.  Brain age prediction based on resting-state functional connectivity patterns using convolutional neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[34]  Andrew T. Drysdale,et al.  Resting-state connectivity biomarkers define neurophysiological subtypes of depression , 2016, Nature Medicine.

[35]  Yufeng Zang,et al.  Standardizing the intrinsic brain: Towards robust measurement of inter-individual variation in 1000 functional connectomes , 2013, NeuroImage.

[36]  Ludovico Minati,et al.  Test‐retest reliability of the default mode network in a multi‐centric fMRI study of healthy elderly: Effects of data‐driven physiological noise correction techniques , 2016, Human brain mapping.

[37]  Marisa O. Hollinshead,et al.  The organization of the human cerebral cortex estimated by intrinsic functional connectivity. , 2011, Journal of neurophysiology.

[38]  Terry K Koo,et al.  A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. , 2016, Journal of chiropractic medicine.

[39]  Jared A. Nielsen,et al.  Multisite functional connectivity MRI classification of autism: ABIDE results , 2013, Front. Hum. Neurosci..

[40]  Bing Chen,et al.  An open science resource for establishing reliability and reproducibility in functional connectomics , 2014, Scientific Data.

[41]  Abraham Z. Snyder,et al.  Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion , 2012, NeuroImage.

[42]  Evan M. Gordon,et al.  Precision Functional Mapping of Individual Human Brains , 2017, Neuron.

[43]  Takashi Hanakawa,et al.  Effects of Field-Map Distortion Correction on Resting State Functional Connectivity MRI , 2017, Front. Neurosci..

[44]  M. Fox,et al.  Individual Variability in Functional Connectivity Architecture of the Human Brain , 2013, Neuron.

[45]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[46]  Erin W. Dickie,et al.  A longitudinal human phantom reliability study of multi-center T1-weighted, DTI, and resting state fMRI data , 2018, Psychiatry Research: Neuroimaging.

[47]  Mert R. Sabuncu,et al.  The influence of head motion on intrinsic functional connectivity MRI , 2012, NeuroImage.

[48]  Theo G. M. van Erp,et al.  Multisite reliability of MR-based functional connectivity , 2017, NeuroImage.

[49]  B. Miller,et al.  Neurodegenerative Diseases Target Large-Scale Human Brain Networks , 2009, Neuron.

[50]  Allan L. Reiss,et al.  209-OR: ADA Presidents' Select Abstract: Type 1 Diabetes and the Developing Brain—A Longitudinal Study of Brain Growth by the Diabetes Research in Children Network (DirecNet) , 2019, Diabetes.

[51]  Gary H. Glover,et al.  Reducing inter-scanner variability of activation in a multicenter fMRI study: Role of smoothness equalization , 2006, NeuroImage.