On the Logic and Purpose of Significance Testing

There has been much recent attention given to the problems involved with the traditional approach to null hypothesis significance testing (NHST). Many have suggested that, perhaps, NHST should be abandoned altogether in favor of other bases for conclusions such as confidence intervals and effect size estimates (e.g., Schmidt, 1996). The purposes of this article are to (a) review the function that data analysis is supposed to serve in the social sciences, (b) examine the ways in which these functions are performed by NHST, (c) examine the case against NHST, and (d) evaluate interval-based estimation as an alternative to NHST.

[1]  J. Mill.  A System of Logic , 1843 .

[2]  H. Jeffreys.  A Treatise on Probability , 1922, Nature.

[3]  E. S. Pearson,et al.  ON THE USE AND INTERPRETATION OF CERTAIN TEST CRITERIA FOR PURPOSES OF STATISTICAL INFERENCE PART I , 1928 .

[4]  Jerzy Neyman,et al.  The testing of statistical hypotheses in relation to probabilities a priori , 1933, Mathematical Proceedings of the Cambridge Philosophical Society.

[5]  M. Kendall,et al.  The Logic of Scientific Discovery. , 1959 .

[6]  W. W. Rozeboom.  The fallacy of the null-hypothesis significance test. , 1960, Psychological bulletin.

[7]  M. Natrella.  The Relation between Confidence Intervals and Tests of Significance: A Teaching Aid , 1960 .

[8]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[9]  D. Bakan,et al.  The test of significance in psychological research. , 1966, Psychological bulletin.

[10]  P. Meehl.  Theory-Testing in Psychology and Physics: A Methodological Paradox , 1967, Philosophy of Science.

[11]  Statistical Issues: A Reader for the Behavioral Sciences , 1972 .

[12]  Joseph H. Danks,et al.  Going beyond tests of significance: Is psychology ready? , 1975 .

[13]  I. Lakatos.  Falsification and the Methodology of Scientific Research Programmes , 1976 .

[14]  Imre Lakatos,et al.  The Methodology of Scientific Research Programmes , 1978 .

[15]  R. P. Carver.  The Case Against Statistical Significance Testing , 1978 .

[16]  H. Putnam,et al.  Reason, Truth and History. , 1983 .

[17]  Ronald C. Serlin,et al.  Rationality in psychological research: The good-enough principle. , 1985 .

[18]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[19]  Richard W. Miller,et al.  Fact and Method. Explanation, Confirmation and Reality in the Natural and the Social Sciences. , 1989 .

[20]  Jacob Cohen,et al.  THINGS I HAVE LEARNED (SO FAR) , 1990 .

[21]  Murray R. Barrick,et al.  THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE: A META-ANALYSIS , 1991 .

[22]  John E. Hunter,et al.  Methods of Meta-Analysis: Correcting Error and Bias in Research Findings , 1991 .

[23]  B. Thompson.  Two and One‐Half Decades of Leadership in Measurement and Evaluation , 1992 .

[24]  Gideon Keren,et al.  A Handbook for data analysis in the behavioral sciences : methodological issues , 1993 .

[25]  Gerd Gigerenzer,et al.  The superego, the ego, and the id in statistical reasoning , 1993 .

[26]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[27]  R. Frick.  Accepting the null hypothesis , 1995, Memory & cognition.

[28]  F. Schmidt.  Statistical Significance Testing and Cumulative Knowledge in Psychology: Implications for Training of Researchers , 1996 .

[29]  I. Kant,et al.  Critique of Pure Reason: Glossary , 1998 .