A method for generating reproducible evidence in fMRI studies

Insights into cognitive neuroscience from neuroimaging techniques are now required to go beyond the localisation of well-known cognitive functions. Fundamental to this is the notion of reproducibility of experimental outcomes. This paper addresses the central issue that functional magnetic resonance imaging (fMRI) experiments will produce more desirable information if researchers begin to search for reproducible evidence rather than only p value significance. The study proposes a methodology for investigating reproducible evidence without conducting separate fMRI experiments. The reproducible evidence is gathered from the separate runs within the study. The associated empirical Bayes and ROC extensions of the linear model provide parameter estimates to determine reproducibility. Empirical applications of the methodology suggest that reproducible evidence is robust to small sample sizes and sensitive to both the magnitude and persistency of brain activation. It is demonstrated that research findings in fMRI studies would be more compelling with supporting reproducible evidence in addition to standard hypothesis testing evidence.

[1]  W. England,et al.  An Exponential Model Used for optimal Threshold selection on ROC Curues , 1988, Medical decision making : an international journal of the Society for Medical Decision Making.

[2]  A. Shmuel,et al.  Sustained Negative BOLD, Blood Flow and Oxygen Consumption Response and Its Coupling to the Positive Response in the Human Brain , 2002, Neuron.

[3]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[4]  R. P. Carver.  The Case Against Statistical Significance Testing, Revisited , 1993 .

[5]  Donald B. Rubin,et al.  Using Empirical Bayes Techniques in the Law School Validity Studies , 1980 .

[6]  M N Branch.  Statistical inference in behavior analysis: Some things significance testing does and does not do , 1999, The Behavior analyst.

[7]  Scott T. Grafton,et al.  Sharing neuroimaging studies of human cognition , 2004, Nature Neuroscience.

[8]  Ari Visa,et al.  Reproducibility of fMRI: Effect of the Use of Contextual Information , 2001, NeuroImage.

[9]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[10]  C. Genovese,et al.  Estimating test‐retest reliability in functional MR imaging I: Statistical methodology , 1997, Magnetic resonance in medicine.

[11]  J. Fell,et al.  Intrasubject reproducibility of presurgical language lateralization and mapping using fMRI , 2003, Neurology.

[12]  Amir Shmuel,et al.  Sustained Negative BOLD and Blood Flow Response and its Coupling to the Positive Response in the Human Brain , 2002 .

[13]  Leslie G. Ungerleider,et al.  Distributed representation of objects in the human ventral visual pathway. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[14]  P. Cheng,et al.  Bridging Functional MR Images and Scientific Inference: Reproducibility Maps , 2003, Journal of Cognitive Neuroscience.

[15]  John C. Gore,et al.  ROC Analysis of Statistical Methods Used in Functional MRI: Individual Subjects , 1999, NeuroImage.

[16]  Karl J. Friston,et al.  The Effects of Presentation Rate During Word and Pseudoword Reading: A Comparison of PET and fMRI , 2000, Journal of Cognitive Neuroscience.

[17]  D. Lindley,et al.  Bayes Estimates for the Linear Model , 1972 .

[18]  Karl J. Friston,et al.  Detecting Activations in PET and fMRI: Levels of Inference and Power , 1996, NeuroImage.

[19]  Leslie G. Ungerleider,et al.  The Representation of Objects in the Human Occipital and Temporal Cortex , 2000, Journal of Cognitive Neuroscience.

[20]  Jonathan D. Cohen,et al.  Reproducibility of fMRI Results across Four Institutions Using a Spatial Working Memory Task , 1998, NeuroImage.

[21]  R. P. Carver.  The Case Against Statistical Significance Testing , 1978 .

[22]  Abraham Z Snyder,et al.  Reliability of functional localization using fMRI , 2003, NeuroImage.

[23]  D C Noll,et al.  Estimating test‐retest reliability in functional MR imaging II: Application to motor and cognitive activation studies , 1997, Magnetic resonance in medicine.

[24]  Lisa A. Best,et al.  Psychology without p values. Data analysis at the turn of the 19th century. , 2000, The American psychologist.

[25]  A. Mechelli,et al.  Neuroimaging Studies of Word and Pseudoword Reading: Consistencies, Inconsistencies, and Limitations , 2003, Journal of Cognitive Neuroscience.

[26]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[27]  Karl J. Friston,et al.  Classical and Bayesian Inference in Neuroimaging: Theory , 2002, NeuroImage.

[28]  R. Nickerson,et al.  Null hypothesis significance testing: a review of an old and continuing controversy. , 2000, Psychological methods.

[29]  G L Shulman,et al.  INAUGURAL ARTICLE by a Recently Elected Academy Member:A default mode of brain function , 2001 .

[30]  Ranjan Maitra,et al.  Test‐retest reliability estimation of functional MRI data , 2002, Magnetic resonance in medicine.

[31]  Lars Kai Hansen,et al.  Optimizing the fMRI data-processing pipeline using prediction and reproducibility performance metrics: I. A preliminary group analysis , 2004, NeuroImage.

[32]  J C Gore,et al.  An roc approach for evaluating functional brain mr imaging and postprocessing protocols , 1995, Magnetic resonance in medicine.

[33]  R. Savoy.  History and future directions of human brain mapping and functional neuroimaging. , 2001, Acta psychologica.

[34]  Alan C. Evans,et al.  A general statistical analysis for fMRI data , 2000, NeuroImage.