Multisite reliability of MR-based functional connectivity

Abstract Recent years have witnessed an increasing number of multisite MRI functional connectivity (fcMRI) studies. While multisite studies provide an efficient way to accelerate data collection and increase sample sizes, especially for rare clinical populations, any effects of site or MRI scanner could ultimately limit power and weaken results. Little data exists on the stability of functional connectivity measurements across sites and sessions. In this study, we assess the influence of site and session on resting state functional connectivity measurements in a healthy cohort of traveling subjects (8 subjects scanned twice at each of 8 sites) scanned as part of the North American Prodrome Longitudinal Study (NAPLS). Reliability was investigated in three types of connectivity analyses: (1) seed‐based connectivity with posterior cingulate cortex (PCC), right motor cortex (RMC), and left thalamus (LT) as seeds; (2) the intrinsic connectivity distribution (ICD), a voxel‐wise connectivity measure; and (3) matrix connectivity, a whole‐brain, atlas‐based approach to assessing connectivity between nodes. Contributions to variability in connectivity due to subject, site, and day‐of‐scan were quantified and used to assess between‐session (test‐retest) reliability in accordance with Generalizability Theory. Overall, no major site, scanner manufacturer, or day‐of‐scan effects were found for the univariate connectivity analyses; instead, subject effects dominated relative to the other measured factors. However, summaries of voxel‐wise connectivity were found to be sensitive to site and scanner manufacturer effects. For all connectivity measures, although subject variance was three times the site variance, the residual represented 60–80% of the variance, indicating that connectivity differed greatly from scan to scan independent of any of the measured factors (i.e., subject, site, and day‐of‐scan). Thus, for a single 5 min scan, reliability across connectivity measures was poor (ICC=0.07–0.17), but increased with increasing scan duration (ICC=0.21–0.36 at 25 min). The limited effects of site and scanner manufacturer support the use of multisite studies, such as NAPLS, as a viable means of collecting data on rare populations and increasing power in univariate functional connectivity studies. However, the results indicate that aggregation of fcMRI data across longer scan durations is necessary to increase the reliability of connectivity estimates at the single‐subject level. HighlightsfcMRI (seed, matrix, ICD) is stable across 8 sites in a Traveling Subjects dataset.No major site, scanner manufacturer, or day‐of‐scan effects were found (GLM).No outlier sites were found (leave‐one‐site‐out analysis of variance).Reliability substantially improves when averaging data over multiple days.Data can be combined across sites to increase power without impacting reliability.