Sample Size in Psychological Research over the Past 30 Years

The American Psychological Association (APA) Task Force on Statistical Inference was formed in 1996 in response to a growing body of research demonstrating methodological issues that threatened the credibility of psychological research, and made recommendations to address them. One issue was the small, even dramatically inadequate, size of samples used in studies published by leading journals. The present study assessed the progress made since the Task Force's final report in 1999. Sample sizes reported in four leading APA journals in 1955, 1977, 1995, and 2006 were compared using nonparametric statistics, while data from the last two waves were fit to a hierarchical generalized linear growth model for more in-depth analysis. Overall, results indicate that the recommendations for increasing sample sizes have not been integrated in core psychological research, although results slightly vary by field. This and other implications are discussed in the context of current methodological critique and practice.

[1]  J. A. Adams,et al.  Psychological bulletin. , 1962, Psychological bulletin.

[2]  C. B. Holmes.  Sample Size in Psychological Research , 1979 .

[3]  Apa Publications,et al.  Reporting standards for research in psychology: why do we need them? What might they be? , 2008, The American psychologist.

[4]  Gregory W. Corder,et al.  Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach , 2009 .

[5]  Effect Size Reporting Practices in Published Articles , 2009 .

[6]  J. Rossi,et al.  Statistical power of psychological research: what have we gained in 20 years? , 1990, Journal of consulting and clinical psychology.

[7]  N. Meyers,et al.  H = W. , 1964, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Yuk Fai Cheong,et al.  HLM 6: Hierarchical Linear and Nonlinear Modeling , 2000 .

[9]  Klaus Krippendorff,et al.  Answering the Call for a Standard Reliability Measure for Coding Data , 2007 .

[10]  J. Ioannidis.  Why Most Published Research Findings Are False , 2005, PLoS medicine.

[11]  C. B. Dobson,et al.  Psychology and the scientific method , 1981 .

[12]  S. Maxwell.  The persistence of underpowered studies in psychological research: causes, consequences, and remedies. , 2004, Psychological methods.

[13]  Fiona Fidler,et al.  The Fifth edition of the Apa Publication Manual: Why its Statistics Recommendations are so Controversial , 2002 .

[14]  Henry L. Roediger,et al.  Research Methods in Psychology , 1985 .

[15]  Eugene B. Zechmeister,et al.  Research methods in psychology, 5th ed. , 2000 .

[16]  J. Rossi,et al.  Statistical power of articles published in three health psychology-related journals. , 2001, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[17]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[18]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[19]  Gerd Gigerenzer,et al.  Do Studies of Statistical Power Have an Effect on the Power of Studies? , 2004 .

[20]  P. F. Merenda.  Psychometrics and Psychometricians in the 20th and 21st Centuries: How it was in the 20th Century and How it is Now , 2007, Perceptual and motor skills.

[21]  Jacob Cohen,et al.  The statistical power of abnormal-social psychological research: a review. , 1962, Journal of abnormal and social psychology.

[22]  J. Rodgers.  The epistemology of mathematical and statistical modeling: a quiet methodological revolution. , 2010, The American psychologist.

[23]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[24]  Sample Size in Non-Apa Journals , 1981 .