Resting-state test–retest reliability of a priori defined canonical networks over different preprocessing steps

Resting-state functional connectivity analysis has become a widely used method for the investigation of human brain connectivity and pathology. The measurement of neuronal activity by functional MRI, however, is impeded by various nuisance signals that reduce the stability of functional connectivity. Several methods exist to address this predicament, but little consensus has yet been reached on the most appropriate approach. Given the crucial importance of reliability for the development of clinical applications, we here investigated the effect of various confound removal approaches on the test–retest reliability of functional-connectivity estimates in two previously defined functional brain networks. Our results showed that gray matter masking improved the reliability of connectivity estimates, whereas denoising based on principal components analysis reduced it. We additionally observed that refraining from using any correction for global signals provided the best test–retest reliability, but failed to reproduce anti-correlations between what have been previously described as antagonistic networks. This suggests that improved reliability can come at the expense of potentially poorer biological validity. Consistent with this, we observed that reliability was proportional to the retained variance, which presumably included structured noise, such as reliable nuisance signals (for instance, noise induced by cardiac processes). We conclude that compromises are necessary between maximizing test–retest reliability and removing variance that may be attributable to non-neuronal sources.

[1]  Simon B. Eickhoff,et al.  Meta-Analytically Informed Network Analysis of Resting State fMRI Reveals Hyperconnectivity in an Introspective Socio-Affective Network in Depression , 2014, PloS one.

[2]  Yul-Wan Sung,et al.  Functional magnetic resonance imaging , 2004, Scholarpedia.

[3]  L. Shah,et al.  Functional magnetic resonance imaging. , 2010, Seminars in roentgenology.

[4]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[5]  Feng Liu,et al.  Abnormal neural activities in first-episode, treatment-naïve, short-illness-duration, and treatment-response patients with major depressive disorder: a resting-state fMRI study. , 2011, Journal of affective disorders.

[6]  Vinod Menon,et al.  Functional connectivity in the resting brain: A network analysis of the default mode hypothesis , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Simon B. Eickhoff,et al.  An improved framework for confound regression and filtering for control of motion artifact in the preprocessing of resting-state functional connectivity data , 2013, NeuroImage.

[8]  K. Zilles,et al.  Coordinate‐based activation likelihood estimation meta‐analysis of neuroimaging data: A random‐effects approach based on empirical estimates of spatial uncertainty , 2009, Human brain mapping.

[9]  Alberto Llera,et al.  ICA-AROMA: A robust ICA-based strategy for removing motion artifacts from fMRI data , 2015, NeuroImage.

[10]  Mary E. Meyerand,et al.  The effect of scan length on the reliability of resting-state fMRI connectivity estimates , 2013, NeuroImage.

[11]  Irene Tracey,et al.  Resting fluctuations in arterial carbon dioxide induce significant low frequency variations in BOLD signal , 2004, NeuroImage.

[12]  S. Ogawa,et al.  Biophysical and Physiological Origins of Blood Oxygenation Level-Dependent fMRI Signals , 2012, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[13]  K Ugurbil,et al.  Functional magnetic resonance imaging of Broca's area during internal speech. , 1993, Neuroreport.

[14]  KENNETH K. KWONGt,et al.  Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation , .

[15]  Scott A. Huettel,et al.  Characterizing individual differences in functional connectivity using dual-regression and seed-based approaches , 2014, NeuroImage.

[16]  Erin K. Molloy,et al.  Using Edge Voxel Information to Improve Motion Regression for rs-fMRI Connectivity Studies , 2015, Brain Connect..

[17]  K. Zilles,et al.  An investigation of the structural, connectional, and functional subspecialization in the human amygdala , 2012, Human brain mapping.

[18]  C. Beckmann,et al.  Spectral characteristics of resting state networks. , 2011, Progress in brain research.

[19]  Steen Moeller,et al.  ICA-based artefact removal and accelerated fMRI acquisition for improved resting state network imaging , 2014, NeuroImage.

[20]  S. Frässle,et al.  Test‐retest reliability of effective connectivity in the face perception network , 2016, Human brain mapping.

[21]  Simon B. Eickhoff,et al.  A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data , 2005, NeuroImage.

[22]  Bin He,et al.  Estimating cortical potentials from scalp EEG's in a realistically shaped inhomogeneous head model by means of the boundary element method. , 1999, IEEE transactions on bio-medical engineering.

[23]  James B. Rowe,et al.  The effect of ageing on fMRI: Correction for the confounding effects of vascular reactivity evaluated by joint fMRI and MEG in 335 adults , 2015, Human brain mapping.

[24]  Yufeng Zang,et al.  Standardizing the intrinsic brain: Towards robust measurement of inter-individual variation in 1000 functional connectomes , 2013, NeuroImage.

[25]  David A. Soltysik,et al.  Improving the use of principal component analysis to reduce physiological noise and motion artifacts to increase the sensitivity of task-based fMRI , 2015, Journal of Neuroscience Methods.

[26]  Dost Öngür,et al.  Anticorrelations in resting state networks without global signal regression , 2012, NeuroImage.

[27]  Catie Chang,et al.  Influence of heart rate on the BOLD signal: The cardiac response function , 2009, NeuroImage.

[28]  Ludovica Griffanti,et al.  Automatic denoising of functional MRI data: Combining independent component analysis and hierarchical fusion of classifiers , 2014, NeuroImage.

[29]  Yong He,et al.  Graph Theoretical Analysis of Functional Brain Networks: Test-Retest Evaluation on Short- and Long-Term Resting-State Functional MRI Data , 2011, PloS one.

[30]  Ewald Moser,et al.  On the origin of respiratory artifacts in BOLD-EPI of the human brain. , 2002, Magnetic resonance imaging.

[31]  Yingli Lu,et al.  Regional homogeneity approach to fMRI data analysis , 2004, NeuroImage.

[32]  Angela R. Laird,et al.  Task- and resting-state functional connectivity of brain regions related to affection and susceptible to concurrent cognitive demand , 2013, NeuroImage.

[33]  M. Raichle,et al.  Disease and the brain's dark energy , 2010, Nature Reviews Neurology.

[34]  David A. Leopold,et al.  The contribution of electrophysiology to functional connectivity mapping , 2013, NeuroImage.

[35]  Rajesh Kumar,et al.  A method for removal of global effects from fMRI time series , 2004, NeuroImage.

[36]  Alan C. Evans,et al.  Structural networks in Alzheimer's disease , 2013, European Neuropsychopharmacology.

[37]  John P. A. Ioannidis,et al.  Empirical evidence for low reproducibility indicates low pre-study odds , 2013, Nature Reviews Neuroscience.

[38]  Xenophon Papademetris,et al.  Graph-theory based parcellation of functional subunits in the brain from resting-state fMRI data , 2010, NeuroImage.

[39]  P. Fox,et al.  Dysregulated left inferior parietal activity in schizophrenia and depression: functional connectivity and characterization , 2013, Front. Hum. Neurosci..

[40]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[41]  Jonathan D. Power,et al.  Recent progress and outstanding issues in motion correction in resting state fMRI , 2015, NeuroImage.

[42]  S. Eickhoff,et al.  Interindividual differences in cognitive flexibility: influence of gray matter volume, functional connectivity and trait impulsivity , 2014, Brain Structure and Function.

[43]  Hang Joon Jo,et al.  Effective Preprocessing Procedures Virtually Eliminate Distance-Dependent Motion Artifacts in Resting State FMRI , 2013, J. Appl. Math..

[44]  Yong He,et al.  Convergence and divergence across construction methods for human brain white matter networks: An assessment based on individual differences , 2015, Human brain mapping.

[45]  Mary E. Meyerand,et al.  The Influence of Physiological Noise Correction on Test-Retest Reliability of Resting-State Functional Connectivity , 2014, Brain Connect..

[46]  Stephen M. Smith,et al.  Advances and Pitfalls in the Analysis and Interpretation of Resting-State FMRI Data , 2010, Front. Syst. Neurosci..

[47]  M. A. Pisauro,et al.  Local and global contributions to hemodynamic activity in mouse cortex , 2016, Journal of neurophysiology.

[48]  Hang Joon Jo,et al.  Mapping sources of correlation in resting state FMRI, with artifact detection and removal , 2010, NeuroImage.

[49]  Xi-Nian Zuo,et al.  Individual Variability and Test-Retest Reliability Revealed by Ten Repeated Resting-State Brain Scans over One Month , 2015, PloS one.

[50]  D. Javitt,et al.  Decreased interhemispheric coordination in schizophrenia: A resting state fMRI study , 2012, Schizophrenia Research.

[51]  R. Cameron Craddock,et al.  A comprehensive assessment of regional variation in the impact of head micromovements on functional connectomics , 2013, NeuroImage.

[52]  Xi-Nian Zuo,et al.  Reliable intrinsic connectivity networks: Test–retest evaluation using ICA and dual regression approach , 2010, NeuroImage.

[53]  V. Haughton,et al.  Frequencies contributing to functional connectivity in the cerebral cortex in "resting-state" data. , 2001, AJNR. American journal of neuroradiology.

[54]  S E Petersen,et al.  Detection of cortical activation during averaged single trials of a cognitive task using functional magnetic resonance imaging. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[55]  Karl J. Friston,et al.  A critique of functional localisers , 2006, NeuroImage.

[56]  Karl J. Friston,et al.  Unified segmentation , 2005, NeuroImage.

[57]  Simon B Eickhoff,et al.  Differential Patterns of Dysconnectivity in Mirror Neuron and Mentalizing Networks in Schizophrenia. , 2016, Schizophrenia bulletin.

[58]  Angela R. Laird,et al.  Tackling the multifunctional nature of Broca's region meta-analytically: Co-activation-based parcellation of area 44 , 2013, NeuroImage.

[59]  Maarten Mennes,et al.  Evaluation of ICA-AROMA and alternative strategies for motion artifact removal in resting state fMRI , 2015, NeuroImage.

[60]  Kevin Murphy,et al.  The impact of global signal regression on resting state correlations: Are anti-correlated networks introduced? , 2009, NeuroImage.

[61]  Maurizio Corbetta,et al.  The human brain is intrinsically organized into dynamic, anticorrelated functional networks. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[62]  Angela R. Laird,et al.  Co-activation patterns distinguish cortical modules, their connectivity and functional differentiation , 2011, NeuroImage.

[63]  R. Turner,et al.  Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[64]  Simon B Eickhoff,et al.  Meta-analysis in human neuroimaging: computational modeling of large-scale databases. , 2014, Annual review of neuroscience.

[65]  Angela R. Laird,et al.  Modelling neural correlates of working memory: A coordinate-based meta-analysis , 2012, NeuroImage.

[66]  Abraham Z. Snyder,et al.  Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion , 2012, NeuroImage.

[67]  Angela R. Laird,et al.  Activation likelihood estimation meta-analysis revisited , 2012, NeuroImage.

[68]  B. Biswal,et al.  Functional connectivity in the motor cortex of resting human brain using echo‐planar mri , 1995, Magnetic resonance in medicine.

[69]  Chaozhe Zhu,et al.  An improved approach to detection of amplitude of low-frequency fluctuation (ALFF) for resting-state fMRI: Fractional ALFF , 2008, Journal of Neuroscience Methods.

[70]  Randy L Buckner,et al.  Human functional connectivity: New tools, unresolved questions , 2010, Proceedings of the National Academy of Sciences.

[71]  Thomas T. Liu,et al.  A component based noise correction method (CompCor) for BOLD and perfusion based fMRI , 2007, NeuroImage.

[72]  John Pluta,et al.  Test–retest stability analysis of resting brain activity revealed by blood oxygen level‐dependent functional MRI , 2012, Journal of magnetic resonance imaging : JMRI.

[73]  M. Fox,et al.  Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging , 2007, Nature Reviews Neuroscience.

[74]  Karl J. Friston,et al.  Movement‐Related effects in fMRI time‐series , 1996, Magnetic resonance in medicine.

[75]  K. Amunts,et al.  Multimodal connectivity mapping of the human left anterior and posterior lateral prefrontal cortex , 2015, Brain Structure and Function.

[76]  Xin Zhang,et al.  Noninvasive cortical imaging of epileptiform activities from interictal spikes in pediatric patients , 2011, NeuroImage.

[77]  Hang Joon Jo,et al.  Trouble at Rest: How Correlation Patterns and Group Differences Become Distorted After Global Signal Regression , 2012, Brain Connect..

[78]  M. Greicius,et al.  Default-Mode Activity during a Passive Sensory Task: Uncoupled from Deactivation but Impacting Activation , 2004, Journal of Cognitive Neuroscience.

[79]  Wenjun Li,et al.  A method to determine the necessity for global signal regression in resting‐state fMRI studies , 2012, Magnetic resonance in medicine.

[80]  B. Biswal,et al.  Simultaneous assessment of flow and BOLD signals in resting‐state functional connectivity maps , 1997, NMR in biomedicine.

[81]  Xiaomu Song,et al.  Data-Driven and Predefined ROI-Based Quantification of Long-Term Resting-State fMRI Reproducibility , 2016, Brain Connect..

[82]  Vivek Prabhakaran,et al.  The effect of resting condition on resting-state fMRI reliability and consistency: A comparison between resting with eyes open, closed, and fixated , 2013, NeuroImage.

[83]  Mert R. Sabuncu,et al.  The influence of head motion on intrinsic functional connectivity MRI , 2012, NeuroImage.

[84]  Rupert Lanzenberger,et al.  Correlations and anticorrelations in resting-state functional connectivity MRI: A quantitative comparison of preprocessing strategies , 2009, NeuroImage.

[85]  Karl J. Friston Functional and Effective Connectivity: A Review , 2011, Brain Connect..

[86]  G. Varoquaux,et al.  Connectivity‐based parcellation: Critique and implications , 2015, Human brain mapping.

[87]  Hang,et al.  Resting-state fMRI can reliably map neural networks in children , 2010 .

[88]  Timothy O. Laumann,et al.  Methods to detect, characterize, and remove motion artifact in resting state fMRI , 2014, NeuroImage.

[89]  P. Fox,et al.  Introspective Minds: Using ALE Meta-Analyses to Study Commonalities in the Neural Correlates of Emotional Processing, Social & Unconstrained Cognition , 2012, PloS one.

[90]  Mark E. Bastin,et al.  Single subject fMRI test–retest reliability metrics and confounding factors , 2013, NeuroImage.

[91]  M. Fox,et al.  The global signal and observed anticorrelated resting state brain networks. , 2009, Journal of neurophysiology.

[92]  Bin He,et al.  Noninvasive Imaging of the High Frequency Brain Activity in Focal Epilepsy Patients , 2014, IEEE Transactions on Biomedical Engineering.

[93]  Angela R. Laird,et al.  Definition and characterization of an extended social-affective default network , 2014, Brain Structure and Function.

[94]  Abraham Z. Snyder,et al.  Function in the human connectome: Task-fMRI and individual differences in behavior , 2013, NeuroImage.

[95]  Qingfei Luo,et al.  Automatic EEG-assisted retrospective motion correction for fMRI (aE-REMCOR) , 2016, NeuroImage.

[96]  Nick F. Ramsey,et al.  Test–retest variability underlying fMRI measurements , 2012, NeuroImage.

[97]  B. Biswal,et al.  The resting brain: unconstrained yet reliable. , 2009, Cerebral cortex.

[98]  Peter A. Bandettini,et al.  Separating respiratory-variation-related fluctuations from neuronal-activity-related fluctuations in fMRI , 2006, NeuroImage.

[99]  Bernard Ng,et al.  Optimization of rs-fMRI Pre-processing for Enhanced Signal-Noise Separation, Test-Retest Reliability, and Group Discrimination , 2015, NeuroImage.