Methods for assessing accuracy and reliability in functional MRI

In this paper, methods for assessing the accuracy and the reliability of functional magnetic resonance imaging techniques are presented. First, a modified receiver operating characteristic analysis is described for evaluating the accuracy of fMRI studies. With this modified approach, the true positives or the activated pixels are estimated based on highly averaged experimental data acquired with the same stimulation/task. Unlike ROC analysis based on simulated activation data, the present approach can be applied to experimentally acquired data without simplifying the activation related changes. To assess the reliability of fMRI studies, the kappa statistic was adopted for evaluating the overall agreement of functional activation maps from repeated experiments in individual subjects. To demonstrate the utility of these techniques, both the ROC analysis and the reliability assessment were applied to quantitatively evaluate the improvement in accuracy and reliability of a retrospective technique for physiological noise reduction in fMRI. © 1997 John Wiley & Sons, Ltd.

[1]  Karl J. Friston,et al.  Movement‐Related effects in fMRI time‐series , 1996, Magnetic resonance in medicine.

[2]  T. L. Davis,et al.  Mr perfusion studies with t1‐weighted echo planar imaging , 1995, Magnetic resonance in medicine.

[3]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[4]  Byron J. T. Morgan,et al.  Some aspects of ROC curve-fitting: Normal and logistic models , 1972 .

[5]  A. Nobre,et al.  Qualitative mapping of cerebral blood flow and functional localization with echo-planar MR imaging and signal targeting with alternating radio frequency. , 1994, Radiology.

[6]  D. Dorfman,et al.  Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data , 1969 .

[7]  D. Tank,et al.  Brain magnetic resonance imaging with contrast dependent on blood oxygenation. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[8]  John A. Swets,et al.  Evaluation of diagnostic systems : methods from signal detection theory , 1982 .

[9]  C. Metz.  Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.

[10]  S. Ogawa,et al.  Oxygenation‐sensitive contrast in magnetic resonance image of rodent brain at high magnetic fields , 1990, Magnetic resonance in medicine.

[11]  Seong-Gi Kim.  Quantification of relative cerebral blood flow change by flow‐sensitive alternating inversion recovery (FAIR) technique: Application to functional mapping , 1995, Magnetic resonance in medicine.

[12]  S. Ogawa.  Brain magnetic resonance imaging with contrast-dependent oxygenation , 1990 .

[13]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[14]  S J Riederer,et al.  Real‐time adaptive motion correction in functional MRI , 1996, Magnetic resonance in medicine.

[15]  Jerome L. Myers,et al.  Implications of latency data for threshold and nonthreshold models of signal detection , 1972 .

[16]  A. Haase,et al.  Rapid NMR Imaging Using Low Flip-Angle Pulses , 2004 .

[17]  X Hu,et al.  Retrospective estimation and correction of physiological fluctuation in functional MRI , 1995, Magnetic resonance in medicine.

[18]  J C Gore,et al.  An roc approach for evaluating functional brain mr imaging and postprocessing protocols , 1995, Magnetic resonance in medicine.

[19]  Ravi S. Menon,et al.  Intrinsic signal changes accompanying sensory stimulation: functional brain mapping with magnetic resonance imaging. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[20]  E C Wong,et al.  Processing strategies for time‐course data sets in functional mri of the human brain , 1993, Magnetic resonance in medicine.

[21]  X. Hu,et al.  Reduction of signal fluctuation in functional MRI using navigator echoes , 1994, Magnetic resonance in medicine.

[22]  S. Ogawa,et al.  Magnetic resonance imaging of blood vessels at high fields: In vivo and in vitro measurements and image simulation , 1990, Magnetic resonance in medicine.

[23]  J. Mazziotta,et al.  Rapid Automated Algorithm for Aligning and Reslicing PET Images , 1992, Journal of computer assisted tomography.

[24]  J A Sorenson,et al.  ROC methods for evaluation of fMRI techniques , 1996, Magnetic resonance in medicine.

[25]  D. McClish.  Analyzing a Portion of the ROC Curve , 1989, Medical decision making : an international journal of the Society for Medical Decision Making.

[26]  X Hu,et al.  Retrospective estimation and correction of physiological artifacts in fMRI by direct extraction of physiological activity from MR data , 1996, Magnetic resonance in medicine.

[27]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .