Single subject fMRI test–retest reliability metrics and confounding factors

While the fMRI test-retest reliability has been mainly investigated from the point of view of group level studies, here we present analyses and results for single-subject test-retest reliability. One important aspect of group level reliability is that not only does it depend on between-session variance (test-retest), but also on between-subject variance. This has partly led to a debate regarding which reliability metric to use and how different sources of noise contribute to between-session variance. Focusing on single subject reliability allows considering between-session only. In this study, we measured test-retest reliability in four behavioural tasks (motor mapping, covert verb generation, overt word repetition, and a landmark identification task) to ensure generalisation of the results and at three levels of data processing (time-series correlation, t value variance, and overlap of thresholded maps) to understand how each step influences the other and how confounding factors influence reliability at each of these steps. The contributions of confounding factors (scanner noise, subject motion, and coregistration) were investigated using multiple regression and relative importance analyses at each step. Finally, to achieve a fuller picture of what constitutes a reliable task, we introduced a bootstrap technique of within- vs. between-subject variance. Our results show that (i) scanner noise and coregistration errors have little contribution to between-session variance (ii) subject motion (especially correlated with the stimuli) can have detrimental effects on reliability (iii) different tasks lead to different reliability results. This suggests that between-session variance in fMRI is mostly caused by the variability of underlying cognitive processes and motion correlated with the stimuli rather than technical limitations of data processing.

[1]  E. Ziegel.  Introduction to Robust Estimation and Hypothesis Testing (2nd ed.) , 2005 .

[2]  Nick F. Ramsey,et al.  Test–retest reliability of fMRI activation during prosaccades and antisaccades , 2007, NeuroImage.

[3]  Nick F. Ramsey,et al.  Test–retest variability underlying fMRI measurements , 2012, NeuroImage.

[4]  Nadine Gaab,et al.  Functional characteristics of developmental dyslexia in left-hemispheric posterior brain regions predate reading onset , 2012, Proceedings of the National Academy of Sciences.

[5]  Karl J. Friston,et al.  Generalisability, Random Effects & Population Inference , 1998, NeuroImage.

[6]  Steven C. R. Williams,et al.  Measuring fMRI reliability with the intra-class correlation coefficient , 2009, NeuroImage.

[7]  Marko Wilke,et al.  An alternative approach towards assessing and accounting for individual motion in fMRI timeseries , 2012, NeuroImage.

[8]  P. Jaccard,et al.  Etude comparative de la distribution florale dans une portion des Alpes et des Jura , 1901 .

[9]  Stephen M Smith,et al.  Fast robust automated brain extraction , 2002, Human brain mapping.

[10]  R. Bowtell,et al.  “sparse” temporal sampling in auditory fMRI , 1999, Human brain mapping.

[11]  R. Wilcox.  Introduction to Robust Estimation and Hypothesis Testing , 1997 .

[12]  Stephen M Smith,et al.  Variability in fMRI: A re‐examination of inter‐session differences , 2005, Human brain mapping.

[13]  P. Sen,et al.  Introduction to bivariate and multivariate analysis , 1981 .

[14]  David J McGonigle,et al.  Variability in FMRI : A Re-Examination of Intersession , 2022 .

[15]  Katrin Amunts,et al.  The human inferior parietal cortex: Cytoarchitectonic parcellation and interindividual variability , 2006, NeuroImage.

[16]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[17]  John Ashburner,et al.  A fast diffeomorphic image registration algorithm , 2007, NeuroImage.

[18]  Satrajit S. Ghosh,et al.  Nipype: A Flexible, Lightweight and Extensible Neuroimaging Data Processing Framework in Python , 2011, Front. Neuroinform..

[19]  J. Fell,et al.  Intrasubject reproducibility of presurgical language lateralization and mapping using fMRI , 2003, Neurology.

[20]  Mark E. Bastin,et al.  Adaptive thresholding for reliable topological inference in single subject fMRI analysis , 2012, Front. Hum. Neurosci..

[21]  Nicholas Ayache,et al.  The Correlation Ratio as a New Similarity Measure for Multimodal Image Registration , 1998, MICCAI.

[22]  H. Ohgaki,et al.  Epidemiology of brain tumors. , 2009, Methods in molecular biology.

[23]  P. Morosan,et al.  Human Primary Auditory Cortex: Cytoarchitectonic Subdivisions and Mapping into a Spatial Reference System , 2001, NeuroImage.

[24]  Simon B. Eickhoff,et al.  Assignment of functional activations to probabilistic cytoarchitectonic areas revisited , 2007, NeuroImage.

[25]  Maria Blatow,et al.  Presurgical Functional MRI in Patients with Brain Tumors , 2007 .

[26]  A. Schleicher,et al.  Two different areas within the primary motor cortex of man , 1996, Nature.

[27]  K. Amunts,et al.  Multimodal architectonic mapping of human superior temporal gyrus , 2005, Anatomy and Embryology.

[28]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[29]  K. McGraw,et al.  Forming inferences about some intraclass correlation coefficients. , 1996 .

[30]  Amos Storkey,et al.  Using a Combination of a Mixture Model and Topological FDR in the Context of Presurgical Planning , 2011 .

[31]  Michael B. Miller,et al.  How reliable are the results from functional magnetic resonance imaging? , 2010, Annals of the New York Academy of Sciences.

[32]  Frank E. Harrell,et al.  A new distribution-free quantile estimator , 1982 .

[33]  Ulrike Groemping,et al.  Relative Importance for Linear Regression in R: The Package relaimpo , 2006 .

[34]  Karl J. Friston,et al.  False discovery rate revisited: FDR and topological inference using Gaussian random fields , 2009, NeuroImage.

[35]  A. Schleicher,et al.  Broca's region revisited: Cytoarchitecture and intersubject variability , 1999, The Journal of comparative neurology.

[36]  Guillaume A. Rousselet,et al.  Improving standards in brain-behavior correlation analyses , 2012, Front. Hum. Neurosci..

[37]  Simon B. Eickhoff,et al.  A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data , 2005, NeuroImage.

[38]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[39]  A. Holmes.  Generalisability, random effects and population inference (abstract) , 1998 .

[40]  B. Dobkin,et al.  Motor Cortex Activation During Treatment May Predict Therapeutic Gains in Paretic Hand Function After Stroke , 2006, Stroke.

[41]  M. Corbetta,et al.  Spatial neglect and attention networks. , 2011, Annual review of neuroscience.

[42]  Joseph T. Devlin,et al.  Consistency and variability in functional localisers , 2009, NeuroImage.

[43]  Simon B. Eickhoff,et al.  Testing anatomically specified hypotheses in functional imaging using cytoarchitectonic maps , 2006, NeuroImage.

[44]  Robert L. Savoy,et al.  Experimental design in brain activation MRI: Cautionary tales , 2005, Brain Research Bulletin.

[45]  Christoph Stippich,et al.  Clinical Functional MRI , 2007 .