Effect Size Guidelines, Sample Size Calculations, and Statistical Power in Gerontology

Abstract Background and Objectives Researchers typically use Cohen’s guidelines of Pearson’s r = .10, .30, and .50, and Cohen’s d = 0.20, 0.50, and 0.80 to interpret observed effect sizes as small, medium, or large, respectively. However, these guidelines were not based on quantitative estimates and are only recommended if field-specific estimates are unknown. This study investigated the distribution of effect sizes in both individual differences research and group differences research in gerontology to provide estimates of effect sizes in the field. Research Design and Methods Effect sizes (Pearson’s r, Cohen’s d, and Hedges’ g) were extracted from meta-analyses published in 10 top-ranked gerontology journals. The 25th, 50th, and 75th percentile ranks were calculated for Pearson’s r (individual differences) and Cohen’s d or Hedges’ g (group differences) values as indicators of small, medium, and large effects. A priori power analyses were conducted for sample size calculations given the observed effect size estimates. Results Effect sizes of Pearson’s r = .12, .20, and .32 for individual differences research and Hedges’ g = 0.16, 0.38, and 0.76 for group differences research were interpreted as small, medium, and large effects in gerontology. Discussion and Implications Cohen’s guidelines appear to overestimate effect sizes in gerontology. Researchers are encouraged to use Pearson’s r = .10, .20, and .30, and Cohen’s d or Hedges’ g = 0.15, 0.40, and 0.75 to interpret small, medium, and large effects in gerontology, and recruit larger samples.

[1]  Alex J Sutton,et al.  Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. , 2008, Journal of clinical epidemiology.

[2]  H. Kivnick,et al.  Is Gerontology in Crisis? , 2015, The Gerontologist.

[3]  Leif D. Nelson,et al.  False-Positive Psychology , 2011, Psychological science.

[4]  G. Cumming Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis , 2011 .

[5]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[6]  Jelte M. Wicherts,et al.  Conducting Meta-Analyses Based on p Values , 2016, Perspectives on psychological science : a journal of the Association for Psychological Science.

[7]  D. Quintana Statistical considerations for reporting and planning heart rate variability case-control studies. , 2017, Psychophysiology.

[8]  Jacob Cohen,et al.  The statistical power of abnormal-social psychological research: a review. , 1962, Journal of abnormal and social psychology.

[9]  Robbie C. M. van Aert,et al.  Meta-analysis using effect size distributions of only statistically significant studies. , 2015, Psychological methods.

[10]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[11]  Gilles E. Gignac,et al.  Effect size guidelines for individual differences researchers , 2016 .

[12]  Thomas Schäfer,et al.  The Meaningfulness of Effect Sizes in Psychological Research: Differences Between Sub-Disciplines and the Impact of Potential Biases , 2019, Front. Psychol..

[13]  Daniël Lakens,et al.  Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs , 2013, Front. Psychol..

[14]  S Duval,et al.  Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis , 2000, Biometrics.

[15]  C. Ferguson,et al.  A Vast Graveyard of Undead Theories , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[16]  J. Wicherts,et al.  The Rules of the Game Called Psychological Science , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[17]  C. Brydges,et al.  A Bayesian Analysis of Evidence in Support of the Null Hypothesis in Gerontological Psychology (or Lack Thereof). , 2019, The journals of gerontology. Series B, Psychological sciences and social sciences.

[18]  S. Maxwell The persistence of underpowered studies in psychological research: causes, consequences, and remedies. , 2004, Psychological methods.

[19]  E. Agadullina,et al.  Empirically Derived Guidelines for Interpreting Effect Size in Social Psychology , 2017 .

[20]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[21]  Wolfgang Viechtbauer,et al.  Conducting Meta-Analyses in R with the metafor Package , 2010 .

[22]  H. Beek F1000Prime recommendation of False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. , 2012 .

[23]  D. Isaacowitz Planning for the Future of Psychological Research on Aging. , 2018, The journals of gerontology. Series B, Psychological sciences and social sciences.

[24]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[25]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[26]  J. Ioannidis Why Most Discovered True Associations Are Inflated , 2008, Epidemiology.

[27]  Frank Renkewitz,et al.  How to detect publication bias in psychological research? A comparative evaluation of six statistical methods , 2019 .

[28]  Leif D. Nelson,et al.  p-Curve and Effect Size , 2014, Perspectives on psychological science : a journal of the Association for Psychological Science.

[29]  R L Levenson Statistical power analysis: implications for researchers, planners, and practitioners in gerontology. , 1980, The Gerontologist.

[30]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[31]  J. Ioannidis,et al.  Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature , 2017, PLoS biology.

[32]  Thomas Boraud,et al.  Low statistical power in biomedical science: a review of three human research domains , 2017, Royal Society Open Science.

[33]  J. Hemphill,et al.  Interpreting the magnitudes of correlation coefficients. , 2003, The American psychologist.