A Guide to Robust Statistical Methods in Neuroscience

There is a vast array of new and improved methods for comparing groups and studying associations that offer the potential for substantially increasing power, providing improved control over the probability of a Type I error, and yielding a deeper and more nuanced understanding of data. These new techniques effectively deal with four insights into when and why conventional methods can be unsatisfactory. But for the non‐statistician, the vast array of new and improved techniques for comparing groups and studying associations can seem daunting, simply because there are so many new methods that are now available. This unit briefly reviews when and why conventional methods can have relatively low power and yield misleading results. The main goal is to suggest some general guidelines regarding when, how, and why certain modern techniques might be used. © 2018 by John Wiley & Sons, Inc.

[1]  Rand R. Wilcox,et al.  Comparing Two Independent Groups Via a Quantile Generalization of the Wilcoxon-Mann-Whitney Test , 2012 .

[2]  Gian Domenico Iannetti,et al.  Whole-Body Mapping of Spatial Acuity for Pain and Touch , 2014, Annals of neurology.

[3]  N. Cliff.  Ordinal methods for behavioral data analysis , 1996 .

[4]  Jie Mi,et al.  Robust Nonparametric Statistical Methods , 1999, Technometrics.

[5]  W. Cleveland.  Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[6]  V. Yohai,et al.  Robust Statistics: Theory and Methods , 2006 .

[7]  Rand R. Wilcox,et al.  Comparing Pearson Correlations: Dealing with Heteroscedasticity and Nonnormality , 2009, Commun. Stat. Simul. Comput..

[8]  J. L. Rasmussen,et al.  Data transformation, Type I error rate and power , 1989 .

[9]  Kjell A. Doksum,et al.  Statistical Tests Based on Transformed Data , 1983 .

[10]  Rand Wilcox,et al.  An inferential method for determining which of two independent variables is most important when there is curvature , 2018, Journal of Modern Applied Statistical Methods.

[11]  Morten W Fagerland,et al.  The Wilcoxon–Mann–Whitney test under scrutiny , 2009, Statistics in medicine.

[12]  Stephane Heritier,et al.  Robust Methods in Biostatistics , 2009 .

[13]  Howard Wainer,et al.  Robust Regression & Outlier Detection , 1988 .

[14]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[15]  B. L. Welch.  THE SIGNIFICANCE OF THE DIFFERENCE BETWEEN TWO MEANS WHEN THE POPULATION VARIANCES ARE UNEQUAL , 1938 .

[16]  Kjell A. Doksum,et al.  Plotting with confidence: Graphical comparisons of two populations , 1976 .

[17]  J. Tukey,et al.  LESS VULNERABLE CONFIDENCE AND SIGNIFICANCE PROCEDURES FOR LOCATION BASED ON A SINGLE SAMPLE : TRIMMING/WINSORIZATION 1 , 2016 .

[18]  Simon J. Sheather,et al.  Confidence intervals based on interpolated order statistics , 1986 .

[19]  J. Ruscio,et al.  A probability-based measure of effect size: robustness to base rates and other factors. , 2008, Psychological methods.

[20]  Joseph P. Romano,et al.  EXACT AND ASYMPTOTICALLY ROBUST PERMUTATION TESTS , 2013, 1304.5939.

[21]  E. Ziegel.  Introduction to Robust Estimation and Hypothesis Testing (2nd ed.) , 2005 .

[22]  S. Sheather,et al.  Robust Estimation & Testing: Staudte/Robust , 1990 .

[23]  J. Ruscio,et al.  Confidence Intervals for the Probability of Superiority Effect Size Measure and the Area Under a Receiver Operating Characteristic Curve , 2012, Multivariate behavioral research.

[24]  Guillaume A. Rousselet,et al.  Robust Correlation Analyses: False Positive and Power Validation Using a New Open Source Matlab Toolbox , 2012, Front. Psychology.

[25]  G. T. Duncan,et al.  A Monte-Carlo study of asymptotically robust tests for correlation coefficients , 1973 .

[26]  D. Grayson,et al.  Some Myths and Legends in Quantitative Psychology , 2004 .

[27]  V. Garovic,et al.  Beyond Bar and Line Graphs: Time for a New Data Presentation Paradigm , 2015, PLoS biology.

[28]  Eric M. Prager,et al.  Reduced GABAergic Inhibition in the Basolateral Amygdala and the Development of Anxiety-Like Behaviors after Mild Traumatic Brain Injury , 2014, PloS one.

[29]  J. Tukey.  The Philosophy of Multiple Comparisons , 1991 .

[30]  K. Yuen,et al.  The two-sample trimmed t for unequal population variances , 1974 .

[31]  H. J. Whitford,et al.  How to Use the Two Sample t‐Test , 1986 .

[32]  R. Newcombe,et al.  Confidence intervals for an effect size measure based on the Mann–Whitney statistic. Part 1: general issues and tail‐area‐based methods , 2006, Statistics in medicine.

[33]  Julia Kastner,et al.  Introduction to Robust Estimation and Hypothesis Testing , 2005 .

[34]  Catherine Lebel,et al.  Reading skill and structural brain development , 2014, Neuroreport.

[35]  Rand R. Wilcox,et al.  Generalized Linear Model Analyses for Treatment Group Equality when Data are Non-Normal , 2016 .

[36]  Curtis L Baker,et al.  Categorically distinct types of receptive fields in early visual cortex. , 2016, Journal of neurophysiology.

[37]  Frank E. Harrell,et al.  A new distribution-free quantile estimator , 1982 .

[38]  J. Brian Gray,et al.  Introduction to Linear Regression Analysis , 2002, Technometrics.

[39]  Joseph P. Romano.  On the behaviour of randomization tests without the group invariance assumption , 1990 .

[40]  Jimmy A. Doi,et al.  A Coverage Probability Approach to Finding an Optimal Binomial Confidence Procedure , 2014 .

[41]  Guillaume A. Rousselet,et al.  Improving standards in brain-behavior correlation analyses , 2012, Front. Hum. Neurosci..

[42]  Barbara J. Wendling,et al.  Woodcock-Johnson III Tests of Achievement. , 2009 .

[43]  Thomas E. Nichols,et al.  luster-based computational methods for mass univariate analyses f event-related brain potentials / fields : A simulation study , 2022 .

[44]  Morton B. Brown,et al.  The Small Sample Behavior of Some Statistics Which Test the Equality of Several Means , 1974 .

[45]  Rand R. Wilcox,et al.  Understanding and Applying Basic Statistical Methods Using R , 2016 .

[46]  A. Hald.  A history of mathematical statistics from 1750 to 1930 , 1998 .

[47]  Subir Ghosh,et al.  Nonparametric Analysis of Longitudinal Data in Factorial Experiments , 2003, Technometrics.

[48]  Y. Hochberg.  A sharper Bonferroni procedure for multiple tests of significance , 1988 .

[49]  Clemens S. Bernhardson,et al.  375: Type I Error Rates When Multiple Comparison Procedures Follow a Significant F Test of ANOVA , 1975 .

[50]  Rand R. Wilcox,et al.  Comparing Measures of Location: Some Small-Sample Results When Distributions Differ in Skewness and Kurtosis Under Heterogeneity of Variances , 2013, Commun. Stat. Simul. Comput..

[51]  S. Sheather,et al.  Robust Estimation and Testing , 1990 .

[52]  G. Hommel.  A stagewise rejective multiple test procedure based on a modified Bonferroni test , 1988 .

[53]  Rand R. Wilcox,et al.  Robust regression: an inferential method for determining which independent variables are most important , 2018, Introduction to Robust Estimation and Hypothesis Testing.

[54]  Cyril R Pernet,et al.  Beyond differences in means: robust graphical methods to compare two groups in neuroscience , 2017, bioRxiv.

[55]  H. Keselman,et al.  Backward, forward and stepwise automated subset selection algorithms: Frequency of obtaining authentic and noise variables , 1992 .

[56]  E. Wagenmakers,et al.  Erroneous analyses of interactions in neuroscience: a problem of significance , 2011, Nature Neuroscience.

[57]  Robert J. Boik,et al.  The Fisher-Pitman permutation test: A non-robust alternative to the normal theory F test when variances are heterogeneous , 1987 .

[58]  Stephen M. Smith,et al.  Permutation inference for the general linear model , 2014, NeuroImage.

[59]  Guillaume A. Rousselet,et al.  A few simple steps to improve the description of group results in neuroscience , 2016, The European journal of neuroscience.

[60]  A. Agresti,et al.  Approximate is Better than “Exact” for Interval Estimation of Binomial Proportions , 1998 .

[61]  Han L. J. van der Maas,et al.  Science Perspectives on Psychological an Agenda for Purely Confirmatory Research on Behalf Of: Association for Psychological Science , 2022 .

[62]  Stacey A. Hancock.  Modern Statistics for the Social and Behavioral Sciences: A Practical Introduction , 2012 .