How Mean is the Mean?

In this paper we voice concerns about the uncritical manner in which the mean is often used as a summary statistic in psychological research. We identify a number of implicit assumptions underlying the use of the mean and argue that the fragility of these assumptions should be more carefully considered. We examine some of the ways in which the potential violation of these assumptions can lead us into significant theoretical and methodological error. Illustrations of alternative models of research already extant within Psychology are used to explore methods of research less mean-dependent and suggest that a critical assessment of the assumptions underlying its use in research play a more explicit role in the process of study design and review.

[1]  D. Balota,et al.  Moving Beyond the Mean in Studies of Mental Chronometry , 2011 .

[2]  Walter Mischel,et al.  Toward an integrative science of the person. , 2004, Annual review of psychology.

[3]  T. Micceri.  The unicorn, the normal curve, and other improbable creatures. , 1989 .

[4]  Joshua Carp,et al.  The secret lives of experiments: Methods reporting in the fMRI literature , 2012, NeuroImage.

[5]  C. Dancey,et al.  Statistics Without Maths For Psychology , 1999 .

[6]  G. M. Reicher.  Perceptual recognition as a function of meaninfulness of stimulus material. , 1969, Journal of experimental psychology.

[7]  R. Hastie,et al.  Hindsight: Biased judgments of past events after the outcomes are known. , 1990 .

[8]  Henry L. Roediger,et al.  Psychology’s Woes and a Partial Cure: The Value of Replication , 2012 .

[9]  W. Uttal.  The New Phrenology: The Limits of Localizing Cognitive Processes in the Brain , 2001 .

[10]  Simon Farrell,et al.  Computational Modeling in Cognition: Principles and Practice , 2010 .

[11]  D. Cramer,et al.  Introduction to Statistics in Psychology , 2005 .

[12]  D. D. Wheeler.  Processes in word recognition , 1970 .

[13]  C. Alberini,et al.  Memory , 2006, Cellular and Molecular Life Sciences CMLS.

[14]  D. Bakan,et al.  The test of significance in psychological research. , 1966, Psychological bulletin.

[15]  George Woodbury,et al.  An Introduction to Statistics , 2001 .

[16]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[17]  Andrew Heathcote,et al.  Linear Deterministic Accumulator Models of Simple Choice , 2012, Front. Psychology.

[18]  John R. Anderson.  Acquisition of cognitive skill. , 1982 .

[19]  E. Boring.  The Logic of the Normal Law of Error in Mental Measurement , 1920 .

[20]  K. Amunts,et al.  Individual variability is not noise , 2013, Trends in Cognitive Sciences.

[21]  G. V. van Orden,et al.  Dispersion of response times reveals cognitive dynamics. , 2009, Psychological review.

[22]  Uri Simonsohn,et al.  Just Post It , 2013, Psychological science.

[23]  Daniel C Molden,et al.  Finding "meaning" in psychology: a lay theories approach to self-regulation, social perception, and social development. , 2006, The American psychologist.

[24]  S. Landau.  Using Survival Analysis in Psychology , 2002 .

[25]  R. Kirk.  Practical Significance: A Concept Whose Time Has Come , 1996 .

[26]  M. R. Novick.  The axioms and principal results of classical test theory , 1965 .

[27]  V. A. Harris,et al.  The Attribution of Attitudes , 1967 .

[28]  Christine P. Dancey,et al.  Statistics Without Maths for Psychology: Using Spss for Windows , 2005 .

[29]  G. Logan.  Toward an instance theory of automatization. , 1988 .

[30]  P. A. Kolers.  Reading A Year Later , 2005 .

[31]  G. Cumming.  Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis , 2011 .

[32]  Jacob Cohen,et al.  THINGS I HAVE LEARNED (SO FAR) , 1990 .

[33]  Lawrence W. Barsalou,et al.  On the vices of nominalization and the virtues of contextualizing , 2010 .

[34]  D. C. Howell.  Statistical Methods for Psychology , 1987 .

[35]  M. Braga,et al.  Exploratory Data Analysis , 2018, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[36]  G. Hammond.  The objections to null hypothesis testing as a means of analysing psychological data , 1996 .

[37]  Eliot R. Smith,et al.  The context principle , 2010 .

[38]  M. R. Novick.  The axioms and principal results of classical test theory , 1965 .

[39]  Kim Kirsner,et al.  Skill acquisition and repetition priming: One principle, many processes , 1996 .

[40]  D. Lewkowicz,et al.  A dynamic systems approach to the development of cognition and action. , 2007, Journal of cognitive neuroscience.

[41]  E. R. Crossman.  A THEORY OF THE ACQUISITION OF SPEED-SKILL∗ , 1959 .

[42]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[43]  K. Kirsner,et al.  Beyond the Learning Curve: The Construction of Mind , 2005 .

[44]  David J. Heeger,et al.  Unreliable Evoked Responses in Autism , 2012, Neuron.

[45]  H. Pashler,et al.  Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition 1 , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[46]  M. Fox,et al.  Individual Variability in Functional Connectivity Architecture of the Human Brain , 2013, Neuron.

[47]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[48]  Scott D. Brown,et al.  The power law repealed: The case for an exponential law of practice , 2000, Psychonomic bulletin & review.

[49]  Allen and Rosenbloom Paul S. Newell,et al.  Mechanisms of Skill Acquisition and the Law of Practice , 1993 .

[50]  R Nathan Spreng,et al.  Functional Brain Changes Following Cognitive and Motor Skills Training , 2013, Neurorehabilitation and neural repair.

[51]  W. Mischel,et al.  A cognitive-affective system theory of personality: reconceptualizing situations, dispositions, dynamics, and invariance in personality structure. , 1995, Psychological review.

[52]  J. Gläscher,et al.  Lesion mapping of cognitive control and value-based decision making in the prefrontal cortex , 2012, Proceedings of the National Academy of Sciences.

[53]  G. Cumming,et al.  Inference by eye: confidence intervals and how to read pictures of data. , 2005, The American psychologist.

[54]  R. Rosenthal.  Experimenter effects in behavioral research , 1968 .

[55]  G. Rousselet,et al.  Single-Trial Analyses: Why Bother? , 2011, Front. Psychology.

[56]  Fernando Marmolejo-Ramos,et al.  Getting the most from your curves: Exploring and reporting data using informative graphical techniques , 2009 .

[57]  P. Meehl.  Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. , 1978 .

[58]  L. Ross.  The Intuitive Psychologist And His Shortcomings: Distortions in the Attribution Process1 , 1977 .

[59]  Stephan R. P. Halloy,et al.  A Theoretical Frameworkfor Abundance Distributions in Complex Systems , 1999 .

[60]  Richard B. Anderson,et al.  Exploring unexplained variation , 2013 .

[61]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[62]  G. Rees.  The new phrenology: the limits of localizing cognitive processes in the brain , 2002 .

[63]  C. McGarty,et al.  Research Methods and Statistics in Psychology , 2003 .