Power, positive predictive value, and sample size calculations for random field theory-based fMRI inference

Recent discussions on the reproducibility of task-related functional magnetic resonance imaging (fMRI) studies have emphasized the importance of power and sample size calculations in fMRI study planning. In general, statistical power and sample size calculations are dependent on the statistical inference framework that is used to test hypotheses. Bibliometric analyses suggest that random field theory (RFT)-based voxel-and cluster-level fMRI inference are the most commonly used approaches for the statistical evaluation of task-related fMRI data. However, general power and sample size calculations for these inference approaches remain elusive. Based on the mathematical theory of RFT-based inference, we here develop power and positive predictive value (PPV) functions for voxel-and cluster-level inference in both uncorrected single test and corrected multiple testing scenarios. Moreover, we apply the theoretical results to evaluate the sample size necessary to achieve desired power and PPV levels based on an fMRI pilot study.

[1]  Karl J. Friston,et al.  Comparing Functional (PET) Images: The Assessment of Significant Change , 1991, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[2]  Martin A. Lindquist,et al.  Ironing out the statistical wrinkles in “ten ironic rules” , 2013, NeuroImage.

[3]  S. Dudoit,et al.  Multiple Hypothesis Testing in Microarray Experiments , 2003 .

[4]  R. Adler,et al.  The Geometry of Random Fields , 1982 .

[5]  Satoru Hayasaka,et al.  Development of PowerMap: a Software Package for Statistical Power Calculation in Neuroimaging Studies , 2012, Neuroinformatics.

[6]  H. Pashler,et al.  Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition 1 , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[7]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[8]  Karl J. Friston,et al.  A unified statistical approach for determining significant signals in images of cerebral activation , 1996, Human brain mapping.

[9]  Karl J. Friston.  Ten ironic rules for non-statistical reviewers , 2012, NeuroImage.

[10]  K. Worsley,et al.  Local Maxima and the Expected Euler Characteristic of Excursion Sets of χ 2, F and t Fields , 1994, Advances in Applied Probability.

[11]  Essa Yacoub,et al.  The WU-Minn Human Connectome Project: An overview , 2013, NeuroImage.

[12]  Jonathan E. Taylor.  A gaussian kinematic formula , 2006, math/0602545.

[13]  Thomas E. Nichols.  Multiple testing corrections, nonparametric methods, and random field theory , 2012, NeuroImage.

[14]  Thomas E. Nichols,et al.  Power calculation for group fMRI studies accounting for arbitrary design and temporal autocorrelation , 2008, NeuroImage.

[15]  Andy Wai Kan Yeung,et al.  An Updated Survey on Statistical Thresholding and Sample Size of fMRI Studies , 2018, Front. Hum. Neurosci..

[16]  J. Borghi,et al.  Data management and sharing in neuroimaging: Practices and perceptions of MRI researchers , 2018, bioRxiv.

[17]  Ruth Seurinck,et al.  Power and sample size calculations for fMRI studies based on the prevalence of active peaks , 2016, bioRxiv.

[18]  J. Ioannidis,et al.  Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature , 2017, PLoS biology.

[19]  Gary H Glover,et al.  Estimating sample size in functional MRI (fMRI) neuroimaging studies: Statistical power analyses , 2002, Journal of Neuroscience Methods.

[20]  Thomas F. Heston,et al.  Predictive power of statistical significance , 2017, World journal of methodology.

[21]  Karl J. Friston,et al.  Assessing the significance of focal activations using their spatial extent , 1994, Human brain mapping.

[22]  D. Ostwald,et al.  Random field theory-based p-values: a review of the SPM implementation , 2018, 1808.04075.

[23]  S. Hayasaka,et al.  Power and sample size calculation for neuroimaging studies by non-central random field theory , 2007, NeuroImage.

[24]  Daniel S. Margulies,et al.  NeuroVault.org: a web-based repository for collecting and sharing unthresholded statistical maps of the human brain , 2014, bioRxiv.

[25]  Karl J. Friston,et al.  Analysis of family‐wise error rates in statistical parametric mapping using random field theory , 2016, Human brain mapping.

[26]  Alan C. Evans,et al.  A Three-Dimensional Statistical Analysis for CBF Activation Studies in Human Brain , 1992, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[27]  Karl J. Friston,et al.  Detecting Activations in PET and fMRI: Levels of Inference and Power , 1996, NeuroImage.

[28]  Dirk Ostwald,et al.  A perceptual decision making EEG/fMRI data set , 2018, bioRxiv.

[29]  Martin A. Lindquist,et al.  Effect Size and Power in fMRI Group Analysis , 2018, bioRxiv.

[30]  Hans Knutsson,et al.  Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates , 2016, Proceedings of the National Academy of Sciences.

[31]  Tor D Wager,et al.  The relation between statistical power and inference in fMRI , 2017, PloS one.

[32]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[33]  Qing Guo,et al.  A systematic review of the reporting of sample size calculations and corresponding data components in observational functional magnetic resonance imaging studies , 2014, NeuroImage.

[34]  Stephen D. Mayhew,et al.  EEG-fMRI Based Information Theoretic Characterization of the Human Perceptual Decision System , 2012, PloS one.

[35]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[36]  J. Ioannidis.  Why Most Published Research Findings Are False , 2019, CHANCE.

[37]  N. Lazar,et al.  Moving to a World Beyond “p < 0.05” , 2019, The American Statistician.

[38]  J. Cao.  The size of the connected components of excursion sets of χ2, t and F fields , 1999, Advances in Applied Probability.

[39]  Denes Szucs,et al.  Correction: Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature , 2016, bioRxiv.

[40]  Anjali Krishnan,et al.  Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations , 2014, NeuroImage.

[41]  K. Worsley,et al.  Detecting Sparse Signals in Random Fields, With an Application to Brain Mapping , 2007 .

[42]  D. Colquhoun.  The False Positive Risk: A Proposal Concerning What to Do About p-Values , 2018, The American Statistician.

[43]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[44]  Thomas E. Nichols,et al.  Best practices in data analysis and sharing in neuroimaging using MRI , 2017, Nature Neuroscience.

[45]  Stephen E. Fienberg,et al.  Testing Statistical Hypotheses , 2005 .

[46]  Joshua Carp,et al.  The secret lives of experiments: Methods reporting in the fMRI literature , 2012, NeuroImage.

[47]  Aron K Barbey,et al.  Small sample sizes reduce the replicability of task-based fMRI studies , 2018, Communications Biology.

[48]  Thomas E. Nichols,et al.  Scanning the horizon: towards transparent and reproducible neuroimaging research , 2016, Nature Reviews Neuroscience.

[49]  S. N. Roy.  On a Heuristic Method of Test Construction and its use in Multivariate Analysis , 1953 .

[50]  Hans Knutsson,et al.  Cluster failure revisited: Impact of first level design and physiological noise on cluster false positive rates , 2018, Human brain mapping.

[51]  Kai Li,et al.  Computational approaches to fMRI analysis , 2017, Nature Neuroscience.

[52]  Jean-Baptiste Poline,et al.  The general linear model and fMRI: Does love last forever? , 2012, NeuroImage.

[53]  R. C. Bose,et al.  Simultaneous Confidence Interval Estimation , 1953 .

[54]  David Colquhoun,et al.  The reproducibility of research and the misinterpretation of p-values , 2017, bioRxiv.

[55]  Nathaniel Rothman,et al.  Assessing the Probability That a Positive Report is False: An Approach for Molecular Epidemiology Studies , 2004 .