Test-retest reliability of the human functional connectome over consecutive days: identifying highly reliable portions and assessing the impact of methodological choices

Countless studies have advanced our understanding of the human brain and its organization by using functional magnetic resonance imaging (fMRI) to derive network representations of human brain function. However, we do not know to what extent these “functional connectomes” are reliable over time. In a large public sample of healthy participants (N = 833) scanned on two consecutive days, we assessed the test-retest reliability of fMRI functional connectivity and the consequences on reliability of three common sources of variation in analysis workflows: atlas choice, global signal regression, and thresholding. By adopting the intraclass correlation coefficient as a metric, we demonstrate that only a small portion of the functional connectome is characterized by good (6–8%) to excellent (0.08–0.14%) reliability. Connectivity between prefrontal, parietal, and temporal areas is especially reliable, but also average connectivity within known networks has good reliability. In general, while unreliable edges are weak, reliable edges are not necessarily strong. Methodologically, reliability of edges varies between atlases, global signal regression decreases reliability for networks and most edges (but increases it for some), and thresholding based on connection strength reduces reliability. Focusing on the reliable portion of the connectome could help quantify brain trait-like features and investigate individual differences using functional neuroimaging.

[1]  Assia Jaillard,et al.  Reliability of graph analysis of resting state fMRI using test-retest dataset from the Human Connectome Project , 2016, NeuroImage.

[2]  Michael W. Cole,et al.  From connectome to cognition: The search for mechanism in human functional brain networks , 2017, NeuroImage.

[3]  Kevin Murphy,et al.  Potential pitfalls when denoising resting state fMRI data using nuisance regression , 2017, NeuroImage.

[4]  Mohamad Adam Bujang,et al.  A simplified guide to determination of sample size requirements for estimating the value of intraclass correlation coefficient: a review , 2017 .

[5]  Essa Yacoub,et al.  The WU-Minn Human Connectome Project: An overview , 2013, NeuroImage.

[6]  J. Lurito,et al.  Correlations in Low-Frequency BOLD Fluctuations Reflect Cortico-Cortical Connections , 2000, NeuroImage.

[7]  Mark Jenkinson,et al.  The minimal preprocessing pipelines for the Human Connectome Project , 2013, NeuroImage.

[8]  Timothy O. Laumann,et al.  Methods to detect, characterize, and remove motion artifact in resting state fMRI , 2014, NeuroImage.

[9]  K. Hwang,et al.  The Contribution of Network Organization and Integration to the Development of Cognitive Control , 2015, PLoS biology.

[10]  Jesper Andersson,et al.  A multi-modal parcellation of human cerebral cortex , 2016, Nature.

[11]  N. Dosenbach,et al.  The frontoparietal network: function, electrophysiology, and importance of individual precision mapping , 2018, Dialogues in clinical neuroscience.

[12]  Ahmad R. Hariri,et al.  General functional connectivity: Shared features of resting-state and task fMRI drive reliable and heritable individual differences in functional brain networks , 2018, NeuroImage.

[13]  Timothy O. Laumann,et al.  Generation and Evaluation of a Cortical Area Parcellation from Resting-State Correlations. , 2016, Cerebral cortex.

[14]  Gang Chen,et al.  Reliability of neural activation and connectivity during implicit face emotion processing in youth , 2018, Developmental Cognitive Neuroscience.

[15]  Patrick G. Bissett,et al.  Uncovering the structure of self-regulation through data-driven ontology discovery , 2019, Nature Communications.

[16]  Jonathan D. Power,et al.  Ridding fMRI data of motion-related influences: Removal of signals with distinct spatial and physical bases in multiecho data , 2018, Proceedings of the National Academy of Sciences.

[17]  Timothy O. Laumann,et al.  Evaluation of Denoising Strategies to Address Motion-Correlated Artifacts in Resting-State Functional Magnetic Resonance Imaging Data from the Human Connectome Project , 2016, Brain Connect..

[18]  Michael W. Cole,et al.  The Frontoparietal Control System , 2014, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[19]  Marisa O. Hollinshead,et al.  The organization of the human cerebral cortex estimated by intrinsic functional connectivity. , 2011, Journal of neurophysiology.

[20]  Ludovica Griffanti,et al.  Automatic denoising of functional MRI data: Combining independent component analysis and hierarchical fusion of classifiers , 2014, NeuroImage.

[21]  M. Banich,et al.  Genetic and environmental influence on the human functional connectome , 2018, bioRxiv.

[22]  L. Williams,et al.  Quantifying person-level brain network functioning to facilitate clinical translation , 2017, Translational Psychiatry.

[23]  D. Cicchetti Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology. , 1994 .

[24]  Chandrasekharan Kesavadas,et al.  Resting state fMRI: A review on methods in resting state connectivity analysis and resting state networks , 2017, The neuroradiology journal.

[25]  Russell A. Poldrack,et al.  Editorial: Reliability and Reproducibility in Functional Connectomics , 2019, Front. Neurosci..

[26]  Anders M. Dale,et al.  An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest , 2006, NeuroImage.

[27]  Danielle S. Bassett,et al.  Evolution of brain network dynamics in neurodevelopment , 2017, Network Neuroscience.

[28]  Peter Kochunov,et al.  Comparison of heritability estimates on resting state fMRI connectivity phenotypes using the ENIGMA analysis pipeline , 2018, Human brain mapping.

[29]  Gang Chen,et al.  Intraclass correlation: improved modeling approaches and applications for neuroimaging , 2017, bioRxiv.

[30]  Yu Zhang,et al.  The Human Brainnetome Atlas: A New Brain Atlas Based on Connectional Architecture , 2016, Cerebral cortex.

[31]  Jonathan D. Power,et al.  Evidence for Hubs in Human Functional Brain Networks , 2013, Neuron.

[32]  Erich P Huang,et al.  Metrology Standards for Quantitative Imaging Biomarkers. , 2015, Radiology.

[33]  X. Zuo,et al.  Test-retest reliabilities of resting-state FMRI measurements in human brain functional connectomics: A systems neuroscience perspective , 2014, Neuroscience & Biobehavioral Reviews.

[34]  D. Yurgelun-Todd,et al.  Reproducibility of Single-Subject Functional Connectivity Measurements , 2011, American Journal of Neuroradiology.

[35]  D. Drachman Do we have brain to spare? , 2005, Neurology.

[36]  M. Breakspear,et al.  The connectomics of brain disorders , 2015, Nature Reviews Neuroscience.

[37]  O. Sporns,et al.  Network neuroscience , 2017, Nature Neuroscience.

[38]  Vince D. Calhoun,et al.  Modern Methods for Interrogating the Human Connectome , 2016, Journal of the International Neuropsychological Society.

[39]  K. McGraw,et al.  Forming inferences about some intraclass correlation coefficients. , 1996 .

[40]  Mary E. Meyerand,et al.  The effect of scan length on the reliability of resting-state fMRI connectivity estimates , 2013, NeuroImage.

[41]  Kevin Murphy,et al.  Towards a consensus regarding global signal regression for resting state functional connectivity MRI , 2017, NeuroImage.

[42]  Annchen R. Knodt,et al.  The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences , 2017, Behavior Research Methods.

[43]  Alan C. Evans,et al.  Meta-Connectomic Analysis Reveals Commonly Disrupted Functional Architectures in Network Modules and Connectors across Brain Disorders , 2018, Cerebral cortex.

[44]  Lawrence Isaac,et al.  Potential pitfalls , 2010, BMJ : British Medical Journal.

[45]  Evan M. Gordon,et al.  Functional System and Areal Organization of a Highly Sampled Individual Human Brain , 2015, Neuron.

[46]  Simon B. Eickhoff,et al.  One-year test–retest reliability of intrinsic connectivity network fMRI in older adults , 2012, NeuroImage.

[47]  Peter A. Bandettini,et al.  Task-based dynamic functional connectivity: Recent findings and open questions , 2017, NeuroImage.

[48]  Dimitri Van De Ville,et al.  The dynamic functional connectome: State-of-the-art and perspectives , 2017, NeuroImage.

[49]  D. Willis A decade on , 2008, Journal of intellectual disabilities : JOID.

[50]  Xi-Nian Zuo,et al.  Harnessing reliability for neuroscience research , 2019, Nature Human Behaviour.

[51]  Dustin Scheinost,et al.  A decade of test-retest reliability of functional connectivity: A systematic review and meta-analysis , 2019, NeuroImage.

[52]  Kristen A. Lindquist,et al.  The brain basis of emotion: A meta-analytic review , 2012, Behavioral and Brain Sciences.

[53]  Olaf Sporns,et al.  Complex network measures of brain connectivity: Uses and interpretations , 2010, NeuroImage.

[54]  Xi-Nian Zuo,et al.  The anatomy of reliability: a must read for future human brain mapping. , 2018, Science bulletin.

[55]  L. Williams,et al.  Defining biotypes for depression and anxiety based on large‐scale circuit dysfunction: a theoretical review of the evidence and future directions for clinical translation , 2017, Depression and anxiety.

[56]  A. Dale,et al.  Whole Brain Segmentation Automated Labeling of Neuroanatomical Structures in the Human Brain , 2002, Neuron.

[57]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.