The Earth Is Not Round (p = .00)

Continued discussion and debate regarding the appropriate use of null hypothesis significance testing (NHST) has led to greater reliance on effect size testing (EST) in published literature. This article examines the myth that uncritical replacement of NHST with EST will improve our science. The use of NHST and EST is described along with a summary of the arguments offered in support and against both. After addressing the veracity of these assertions, the article describes the concept of the translation mechanism and compares the success of NHST and EST as mechanisms. Finally, the authors suggest changes that may facilitate translation in future research.

[1]  E. S. Pearson,et al.  ON THE USE AND INTERPRETATION OF CERTAIN TEST CRITERIA FOR PURPOSES OF STATISTICAL INFERENCE PART I , 1928 .

[2]  Jerzy Neyman,et al.  The testing of statistical hypotheses in relation to probabilities a priori , 1933, Mathematical Proceedings of the Cambridge Philosophical Society.

[3]  Henry S. Leonard,et al.  Testability and Meaning. , 1937 .

[4]  Willard Van Orman Quine,et al.  Word and Object , 1960 .

[5]  T. Kuhn,et al.  The Structure of Scientific Revolutions , 1963 .

[6]  P. Meehl.  Theory-Testing in Psychology and Physics: A Methodological Paradox , 1967, Philosophy of Science.

[7]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[8]  P. Meehl.  Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. , 1978 .

[9]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[10]  Ronald C. Serlin,et al.  Rationality in psychological research: The good-enough principle. , 1985 .

[11]  Richard W. Miller.  Fact and Method: Explanation, Confirmation and Reality in the Natural and the Social Sciences , 1988 .

[12]  John E. Hunter,et al.  Methods of Meta-Analysis: Correcting Error and Bias in Research Findings , 1991 .

[13]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[14]  R. Falk,et al.  Significance Tests Die Hard , 1995 .

[15]  R. Frick.  Accepting the null hypothesis , 1995, Memory & cognition.

[16]  F. Schmidt.  Statistical Significance Testing and Cumulative Knowledge in Psychology: Implications for Training of Researchers , 1996 .

[17]  W. Dunlap,et al.  On the Logic and Purpose of Significance Testing , 1997 .

[18]  Patrick E. Shrout,et al.  Should Significance Tests be Banned? Introduction to a Special Section Exploring the Pros and Cons , 1997 .

[19]  J. Hunter.  Needed: A Ban on the Significance Test , 1997 .

[20]  J. M. Cortina,et al.  Determining relative importance of predictors with the observational design. , 1998 .

[21]  Hossein Nouri,et al.  Effect Size for ANOVA Designs , 1999 .

[22]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[23]  Brett Myors,et al.  Testing the hypothesis that treatments have negligible effects : Minimum-effect tests in the general linear model , 1999 .

[24]  R. Nickerson,et al.  Null hypothesis significance testing: a review of an old and continuing controversy. , 2000, Psychological methods.

[25]  B. Thompson.  Significance, Effect Sizes, Stepwise Methods, and Other Issues: Strong Arguments Move the Field , 2001 .

[26]  J. Hemphill,et al.  Interpreting the magnitudes of correlation coefficients. , 2003, The American psychologist.

[27]  Rex B. Kline,et al.  Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research , 2004 .

[28]  H. Stam.  Pythagoreanism, meaning and the appeal to number , 2006 .

[29]  T. Levine,et al.  A Communication Researchers' Guide to Null Hypothesis Significance Testing and Alternatives. , 2008 .

[30]  Marcello Gallucci,et al.  A conceptual and empirical examination of justifications for dichotomization. , 2009, Psychological methods.

[31]  Charles E. Lance,et al.  Statistical and methodological myths and urban legends : doctrine, verity, and fable in the organizational and social sciences , 2009 .

[32]  R. Landis,et al.  When small effect sizes tell a big story, and when large effect sizes don't. , 2009 .

[33]  Julian N. Marewski,et al.  Beyond the null ritual : Formal modeling of psychological processes , 2009 .

[34]  J. Edwards,et al.  The Presence of Something or the Absence of Nothing: Increasing Theoretical Precision in Management Research , 2010 .

[35]  J. Rodgers.  The epistemology of mathematical and statistical modeling: a quiet methodological revolution. , 2010, The American psychologist.