Beyond differences in means: robust graphical methods to compare two groups in neuroscience

If many changes are necessary to improve the quality of neuroscience research, one relatively simple step could have great pay‐offs: to promote the adoption of detailed graphical methods, combined with robust inferential statistics. Here, we illustrate how such methods can lead to a much more detailed understanding of group differences than bar graphs and t‐tests on means. To complement the neuroscientist's toolbox, we present two powerful tools that can help us understand how groups of observations differ: the shift function and the difference asymmetry function. These tools can be combined with detailed visualisations to provide complementary perspectives about the data. We provide implementations in R and MATLAB of the graphical tools, and all the examples in the article can be reproduced using R scripts.

[1]  John P. A. Ioannidis,et al.  A manifesto for reproducible science , 2017, Nature Human Behaviour.

[2]  David M Erceg-Hurn,et al.  Modern robust statistical methods: an easy way to maximize the accuracy and power of your research. , 2008, The American psychologist.

[3]  Curtis L Baker,et al.  Categorically distinct types of receptive fields in early visual cortex. , 2016, Journal of neurophysiology.

[4]  Guillaume A. Rousselet,et al.  A few simple steps to improve the description of group results in neuroscience , 2016, The European journal of neuroscience.

[5]  Guillaume A. Rousselet,et al.  A statistical framework for neuroimaging data analysis based on mutual information estimated via a gaussian copula , 2016, bioRxiv.

[6]  E. Lauber,et al.  Conditional and unconditional automaticity: a dual-process model of effects of spatial stimulus-response correspondence. , 1994, Journal of experimental psychology. Human perception and performance.

[7]  Rand R. Wilcox,et al.  Comparing two dependent groups via quantiles , 2012 .

[8]  E. Wagenmakers,et al.  Detecting and avoiding likely false‐positive findings – a practical guide , 2017, Biological reviews of the Cambridge Philosophical Society.

[9]  T. Bjerkedal.  Acquisition of resistance in guinea pigs infected with different doses of virulent tubercle bacilli. , 1960, American journal of hygiene.

[10]  C. Wilke.  Streamlined Plot Theme and Plot Annotations for 'ggplot2' , 2015 .

[11]  Kjell A. Doksum,et al.  Empirical Probability Plots and Statistical Inference for Nonlinear Models in the Two-Sample Case , 1974 .

[12]  Rand R. Wilcox,et al.  Measuring effect size: A non-parametric analogue of ω2 , 1999 .

[13]  Rand R. Wilcox,et al.  Graphical Methods for Assessing Effect Size: Some Alternatives to Cohen's d , 2006 .

[14]  Han L. J. van der Maas,et al.  Science Perspectives on Psychological an Agenda for Purely Confirmatory Research on Behalf Of: Association for Psychological Science , 2022 .

[15]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[16]  Jean Rouat,et al.  Comparative effects of adaptation on layers II–III and V–VI neurons in cat V1 , 2016, The European journal of neuroscience.

[17]  Rand R. Wilcox,et al.  Comparing Two Independent Groups Via Multiple Quantiles , 1995 .

[18]  E. Ziegel.  Introduction to Robust Estimation and Hypothesis Testing (2nd ed.) , 2005 .

[19]  Guillaume A. Rousselet,et al.  A robust and representative lower bound on object processing speed in humans , 2015, The European journal of neuroscience.

[20]  David Colquhoun,et al.  An investigation of the false discovery rate and the misinterpretation of p-values , 2014, Royal Society Open Science.

[21]  E. Paulesu,et al.  When all hypotheses are right: A multifocal account of dyslexia , 2009, Human brain mapping.

[22]  Cyril R Pernet,et al.  Brain classification reveals the right cerebellum as the best biomarker of dyslexia , 2009, BMC Neuroscience.

[23]  Dianne Cook,et al.  Data Visualization and Statistical Graphics in Big Data Analysis , 2016 .

[24]  Pedro M. Valero-Mora,et al.  ggplot2: Elegant Graphics for Data Analysis , 2010 .

[25]  Lingxin Hao,et al.  Relative Distribution Methods , 2010 .

[26]  Frank E. Harrell,et al.  A new distribution-free quantile estimator , 1982 .

[27]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[28]  K. Doksum,et al.  Some graphical methods in statistics: A review and some extensions , 1977 .

[29]  Guillaume A. Rousselet,et al.  Beyond differences in means: robust graphical methods to compare two groups in neuroscience , 2017 .

[30]  Erik B. Erhardt,et al.  Data Visualization in the Neurosciences: Overcoming the Curse of Dimensionality , 2012, Neuron.

[31]  Thomas E. Nichols,et al.  Scanning the horizon: towards transparent and reproducible neuroimaging research , 2016, Nature Reviews Neuroscience.

[32]  Taxiarchis Botsis,et al.  Seeing is believing: good graphic design principles for medical research , 2015, Statistics in medicine.

[33]  Richard D. Morey,et al.  Delta Plots and Coherent Distribution Ordering , 2008 .

[34]  Rand R. Wilcox,et al.  Comparing two independent groups via the lower and upper quantiles , 2014 .

[35]  Rand R. Wilcox,et al.  Modern Insights About Pearson’s Correlation and Least Squares Regression , 2001 .

[36]  Rand R. Wilcox,et al.  Basic Statistics: Understanding Conventional Methods and Modern Insights , 2009 .

[37]  R. Wilcox,et al.  Measuring effect size: a non-parametric analogue of omega 2. , 1999, The British journal of mathematical and statistical psychology.

[38]  Christopher Rao,et al.  Graphs in Statistical Analysis , 2010 .

[39]  Kjell A. Doksum,et al.  Plotting with confidence: Graphical comparisons of two populations , 1976 .

[40]  Rand R. Wilcox,et al.  Comparing Two Independent Groups Via a Quantile Generalization of the Wilcoxon-Mann-Whitney Test , 2012 .

[41]  K Richard Ridderinkhof,et al.  Delta plots in the study of individual differences: new tools reveal response inhibition deficits in AD/Hd that are eliminated by methylphenidate treatment. , 2005, Journal of abnormal psychology.

[42]  Jeffrey N Rouder,et al.  Exploring the differences in distributional properties between Stroop and Simon effects using delta plots , 2010, Attention, perception & psychophysics.

[43]  H. Keselman,et al.  Modern robust data analysis methods: measures of central tendency. , 2003, Psychological methods.

[44]  Julia Kastner,et al.  Introduction to Robust Estimation and Hypothesis Testing , 2005 .

[45]  V. Garovic,et al.  Beyond Bar and Line Graphs: Time for a New Data Presentation Paradigm , 2015, PLoS biology.