The principled control of false positives in neuroimaging.

An incredible amount of data is generated in the course of a functional neuroimaging experiment. The quantity of data gives us improved temporal and spatial resolution with which to evaluate our results. It also creates a staggering multiple testing problem. A number of methods have been created that address the multiple testing problem in neuroimaging in a principled fashion. These methods place limits on either the familywise error rate (FWER) or the false discovery rate (FDR) of the results. These principled approaches are well established in the literature and are known to properly limit the amount of false positives across the whole brain. However, a minority of papers are still published every month using methods that are improperly corrected for the number of tests conducted. These latter methods place limits on the voxelwise probability of a false positive and yield no information on the global rate of false positives in the results. In this commentary, we argue in favor of a principled approach to the multiple testing problem--one that places appropriate limits on the rate of false positives across the whole brain gives readers the information they need to properly evaluate the results.

[1]  J. Mumford,et al.  Independence in ROI analysis: where is the voodoo? , 2009, Social cognitive and affective neuroscience.

[2]  S M Kosslyn,et al.  If neuroimaging is the answer, what is the question? , 1999, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[3]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[4]  M. Mirmehdi,et al.  16th Annual Meeting of the Organization for Human Brain Mapping , 2010 .

[5]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[6]  Em Mead,et al.  Society for Neuroscience Annual Meeting , 2009 .

[7]  N. Kanwisher,et al.  Does the fusiform face area contain subregions highly selective for nonfaces? , 2007, Nature Neuroscience.

[8]  A. Gelman,et al.  Correlations and Multiple Comparisons in Functional Imaging: A Statistical Perspective (Commentary on Vul et al., 2009) , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[9]  Thomas E. Nichols,et al.  Simple group fMRI modeling and inference , 2009, NeuroImage.

[10]  N. Taleb Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets , 2001 .

[11]  Michael B. Miller,et al.  of Serendipitous and Unexpected Results Neural Correlates of Interspecies Perspective Taking in the Post-Mortem Atlantic Salmon : An Argument For Proper Multiple Comparisons Correction , 2010 .

[12]  R. Desimone,et al.  Neural Mechanisms of Visual Working Memory in Prefrontal Cortex of the Macaque , 1996, The Journal of Neuroscience.

[13]  Alan C. Evans,et al.  A Three-Dimensional Statistical Analysis for CBF Activation Studies in Human Brain , 1992, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[14]  Jonathan D. Cohen,et al.  Improved Assessment of Significant Activation in Functional Magnetic Resonance Imaging (fMRI): Use of a Cluster‐Size Threshold , 1995, Magnetic resonance in medicine.

[15]  Brian Knutson,et al.  Valence and salience contribute to nucleus accumbens activation , 2008, NeuroImage.

[16]  R. Wootton,et al.  Structural colour: Now you see it — now you don't , 2001, Nature.

[17]  CM Bennett,et al.  Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: an argument for multiple comparisons correction , 2009, NeuroImage.

[18]  D. W Loring,et al.  Now you see it, now you don’t: statistical and methodological considerations in fMRI , 2002, Epilepsy & Behavior.

[19]  K. McDermott,et al.  Creating false memories: Remembering words not presented in lists. , 1995 .

[20]  R. Rosenthal The file drawer problem and tolerance for null results , 1979 .

[21]  Thomas E. Nichols,et al.  Controlling the familywise error rate in functional neuroimaging: a comparative review , 2003, Statistical methods in medical research.

[22]  John D. Storey,et al.  Statistical significance for genomewide studies , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Jonathan S. Cant,et al.  Crinkling and crumpling: An auditory fMRI study of material properties , 2008, NeuroImage.

[24]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[25]  Daniel L. Schacter,et al.  Neuroanatomical Correlates of Veridical and Illusory Recognition Memory: Evidence from Positron Emission Tomography , 1996, Neuron.