Mass univariate analysis of event-related brain potentials/fields II: Simulation studies.

Mass univariate analysis is a relatively new approach for the study of ERPs/ERFs. It consists of many statistical tests and one of several powerful corrections for multiple comparisons. Multiple comparison corrections differ in their power and permissiveness. Moreover, some methods are not guaranteed to work or may be overly sensitive to uninteresting deviations from the null hypothesis. Here we report the results of simulations assessing the accuracy, permissiveness, and power of six popular multiple comparison corrections (permutation-based control of the familywise error rate [FWER], weak control of FWER via cluster-based permutation tests, permutation-based control of the generalized FWER, and three false discovery rate control procedures) using realistic ERP data. In addition, we look at the sensitivity of permutation tests to differences in population variance. These results will help researchers apply and interpret these procedures.

[1]  Kyung In Kim,et al.  Effects of dependence in high-dimensional multiple testing problems , 2008, BMC Bioinformatics.

[2]  J. Pernier,et al.  ERP Manifestations of Processing Printed Words at Different Psycholinguistic Levels: Time Course and Scalp Distribution , 1999, Journal of Cognitive Neuroscience.

[3]  John Suckling,et al.  Global, voxel, and cluster tests, by theory and permutation, for a difference between two groups of structural MR images of the brain , 1999, IEEE Transactions on Medical Imaging.

[4]  S. Weiss,et al.  New concepts of multiple tests and their use for evaluating high-dimensional EEG data , 2005, Journal of Neuroscience Methods.

[5]  Eduardo Martínez-Montes,et al.  False discovery rate and permutation test: An evaluation in ERP data analysis , 2010, Statistics in medicine.

[6]  B. Murphy,et al.  Some two-sample tests when the variances are unequal: a simulation study. , 1967, Biometrika.

[7]  Marta Kutas,et al.  The phonemic restoration effect reveals pre-N400 effect of supportive sentence context in speech perception , 2010, Brain Research.

[8]  B. Efron.  Large-Scale Simultaneous Hypothesis Testing , 2004 .

[9]  Sabine Weiss,et al.  Multivariate tests for the evaluation of high-dimensional EEG data , 2004, Journal of Neuroscience Methods.

[10]  I. Winkler,et al.  The concept of auditory stimulus representation in cognitive neuroscience. , 1999, Psychological bulletin.

[11]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[12]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[13]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[14]  R. Blair,et al.  An alternative method for significance testing of waveform difference potentials. , 1993, Psychophysiology.

[15]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[16]  R. Simon,et al.  Controlling the number of false discoveries: application to high-dimensional genomic data , 2004 .

[17]  Zijiang J. He,et al.  Vertical and horizontal references determined by linear perspective and optic flow information , 2010 .

[18]  Y. Benjamini,et al.  Adaptive linear step-up procedures that control the false discovery rate , 2006 .

[19]  P. Hall,et al.  Robustness of multiple testing procedures against dependence , 2009, 0903.0464.

[20]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[21]  S. Luck.  An Introduction to the Event-Related Potential Technique , 2005 .

[22]  David M. Groppe,et al.  Mass univariate analysis of event-related brain potentials/fields I: a critical tutorial review. , 2011, Psychophysiology.

[23]  S. Hillyard,et al.  Electrical Signs of Selective Attention in the Human Brain , 1973, Science.

[24]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[25]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[26]  Bruno A Olshausen,et al.  Timecourse of neural signatures of object recognition. , 2003, Journal of vision.