Statistical power of psychological research: what have we gained in 20 years?

Power was calculated for 6,155 statistical tests in 221 journal articles published in the 1982 volumes of the Journal of Abnormal Psychology, Journal of Consulting and Clinical Psychology, and Journal of Personality and Social Psychology. Power to detect small, medium, and large effects was .17, .57, and .83, respectively. 20 years after Cohen (1962) conducted the first power survey, the power of psychological research is still low. The implications of these results concerning the proliferation of Type I errors in the published literature, the failure of replication studies, and the interpretation of null (negative) results are emphasized. An example is given of the use of power analysis to help interpret null results by setting probable upper bounds on the magnitudes of effects. Limitations of statistical power analyses, suggestions for future research, sources of computational information, and recommendations for improving power are discussed.

[1]  T. Sterling.  Publication Decisions and their Possible Effects on Inferences Drawn from Tests of Significance—or Vice Versa , 1959 .

[2]  L. J. Chase,et al.  Communication disorders: a power analytic assessment of recent research. , 1975, Journal of communication disorders.

[3]  J. Bones.  Statistical Power Analysis And Geography1 , 1972 .

[4]  James P. Stevens,et al.  Power of the multivariate analysis of variance tests. , 1980 .

[5]  Masoud Hemmasi,et al.  Assessment of statistical power in contemporary strategy research , 1987 .

[6]  P. Muchinsky.  Some changes in the characteristics of articles published in the Journal of Applied Psychology over the past 20 years. , 1979 .

[7]  Statistical power in biological psychiatry , 1981, Psychiatry Research.

[8]  Stanley J. Baran,et al.  An Assessment of Quantitative Research in Mass Communication , 1976 .

[9]  Harris Cooper,et al.  Expected Effect Sizes , 1982 .

[10]  Richard M. Tolman,et al.  The Statistical Power of a Decade of Social Work Education Research , 1986, Social Service Review.

[11]  L. J. Chase,et al.  A power‐analytic examination of contemporary communication research , 1975 .

[12]  D. S. King.  Statistical power of the controlled research on wheat gluten and schizophrenia , 1985, Biological Psychiatry.

[13]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[14]  Joseph S. Rossi,et al.  Tables of Effect Size for Z Score Tests of Differences between Proportions and between Correlation Coefficients , 1985 .

[15]  James K. Brewer,et al.  A NOTE ON THE POWER OF STATISTICAL TESTS IN THE JOURNAL OF EDUCATIONAL MEASUREMENT , 1973 .

[16]  N. Laubscher,et al.  Normalizing the Noncentral $t$ and $F$ Distributions , 1960 .

[17]  A. Greenwald.  Consequences of Prejudice Against the Null Hypothesis , 1975 .

[18]  James K. Brewer,et al.  On the Power of Statistical Tests in the American Educational Research Journal 1 , 1972 .

[19]  Alan G. Sawyer,et al.  Statistical Power and Effect Size in Marketing Research , 1981 .

[20]  L. J. Chase,et al.  Statistical Power: Derivation, Development, and Data-Analytic Implications , 1976 .

[21]  Jacob Cohen,et al.  Statistical power analysis : a computer program , 1988 .

[22]  Alan Stuart,et al.  Data-Dredging Procedures in Survey Analysis , 1966 .

[23]  J E Christensen,et al.  Statistical power analysis of health, physical education, and recreation research. , 1977, Research quarterly.

[24]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[25]  B J Jones,et al.  An analysis of the power of statistical tests reported in the Research Quarterly. , 1972, Research quarterly.

[26]  A. Tversky,et al.  BELIEF IN THE LAW OF SMALL NUMBERS , 1971, Pediatrics.

[27]  B. Wolman,et al.  Handbook of clinical psychology , 1965 .

[28]  P. Lachenbruch,et al.  Design Sensitivity: Statistical Power for Experimental Research. , 1989 .

[29]  D. Bakan,et al.  The test of significance in psychological research. , 1966, Psychological bulletin.

[30]  J. Nee,et al.  A Comparison of Two Noncentral F Approximations, with Applications to Power Analysis in Set Correlation. , 1987, Multivariate behavioral research.

[31]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[32]  N. Nie.  SPSS-X User's Guide , 1983 .

[33]  Mark W. Lipsey,et al.  Design Sensitivity: Statistical Power for Experimental Research. , 1989 .

[34]  C. B. Holmes.  Sample Size in Psychological Research , 1979 .

[35]  A. Kazdin,et al.  Power to detect differences between alternative treatments in comparative psychotherapy outcome research. , 1989, Journal of consulting and clinical psychology.

[36]  G. Gigerenzer,et al.  Do studies of statistical power have an effect on the power of studies , 1989 .

[37]  Jeffrey Katzer,et al.  An Analysis of the Use of Statistical Testing in Communication Research , 1973 .

[38]  John A. Daly,et al.  Statistical Power in Research in English Education. , 1983 .

[39]  Robert P. Abelson,et al.  A Variance Explanation Paradox: When a Little is a Lot , 1985 .

[40]  L. J. Chase,et al.  Statistical power in physical anthropology: a technical report. , 1978, American journal of physical anthropology.

[41]  Donald B. Rubin,et al.  A Simple, General Purpose Display of Magnitude of Experimental Effect , 1982 .

[42]  Jacob Cohen,et al.  The statistical power of abnormal-social psychological research: a review. , 1962, Journal of abnormal and social psychology.

[43]  John E. Overall.  Classical statistical hypotheses testing within the context of Bayesian theory. , 1969 .

[44]  Richard F. Haase.  Power Analysis of Research in Counselor Education , 1974 .

[45]  N E Breslow,et al.  Power considerations in epidemiologic studies of vinyl chloride workers. , 1981, American journal of epidemiology.

[46]  John G. Orme,et al.  Statistical power and Type II errors in social work research , 1986 .

[47]  J. Stevens.  Applied Multivariate Statistics for the Social Sciences , 1986 .

[48]  Richard Goldstein.  Power and Sample Size via MS/PC-DOS Computers , 1989 .

[49]  R. Katzell,et al.  Differential validity revived. , 1977 .

[50]  M. Abramowitz,et al.  Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55) , 1965 .

[51]  Richard F. Haase,et al.  How significant is a significant difference? Average effect size of research in counseling psychology. , 1982 .

[52]  M. Cowles,et al.  On the Origins of the .05 Level of Statistical Significance , 1982 .

[53]  P. Meehl.  Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. , 1978 .

[54]  R. Rosenthal,et al.  Statistical Procedures and the Justification of Knowledge in Psychological Science , 1989 .

[55]  H. Kraemer,et al.  How Many Subjects? Statistical Power Analysis in Research. , 1989 .

[56]  M. Moore.  Empirical Statistics: III. Changes in Use of Statistics in Psychology , 1981 .

[57]  Thomas W. Woolley,et al.  A follow-up power analysis of the statistical tests used in the Journal of Research in Science Teaching , 1983 .

[58]  T C Chalmers,et al.  The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 "negative" trials. , 1978, The New England journal of medicine.

[59]  L. J. Chase,et al.  A statistical power analysis of applied psychological research. , 1976 .

[60]  Milton Abramowitz,et al.  Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables , 1964 .

[61]  John E. Penick,et al.  The power of statistical tests in science teachnig research , 1972 .

[62]  R L Levenson.  Statistical power analysis: implications for researchers, planners, and practitioners in gerontology. , 1980, The Gerontologist.

[63]  Kenneth J. Ottenbacher,et al.  Statistical Power and Research in Occupational Therapy , 1982 .

[64]  Masoud Hemmasi,et al.  Statistical Power in Contemporary Management Research , 1987 .