Significance test or effect size

I describe and question the argument that in psychological research, the significance test should be replaced (or, at least, supplemented) by a more informative index (viz., effect size or statistical power) in the case of theory-corroboration experimentation because it has been made on the basis of some debatable assumptions about the rationale of scientific investigation. The rationale of theory-corroboration experimentation requires nothing more than a binary decision about the relation between two variables. This binary decision supplies the minor premise for the syllogism implicated when a theory is being tested. Some metatheoretical considerations reveal that the magnitude of the effect-size estimate is not a satisfactory alternative to the significance test.

[1]  J. L. Myers.  Fundamentals of Experimental Design , 1972 .

[2]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[3]  D. Bakan,et al.  The test of significance in psychological research. , 1966, Psychological bulletin.

[4]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[5]  K. Popper,et al.  Conjectures and Refutations , 1963 .

[6]  Donald B. Rubin,et al.  A Simple, General Purpose Display of Magnitude of Experimental Effect , 1982 .

[7]  D. P. Hartmann,et al.  Meta-analysis: Techniques, applications, and functions. , 1983 .

[8]  P. Meehl.  Theory-Testing in Psychology and Physics: A Methodological Paradox , 1967, Philosophy of Science.

[9]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[10]  D W Fiske,et al.  The meta-analytic revolution in outcome research. , 1983, Journal of consulting and clinical psychology.

[11]  D. Lykken.  Statistical significance in psychological research. , 1968, Psychological bulletin.

[12]  D. P. Hartmann,et al.  A critical appraisal of meta-analysis. , 1982, The British journal of clinical psychology.

[13]  P. Meehl.  Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. , 1978 .

[14]  Donald B. Rubin,et al.  A Note on Percent Variance Explained as A Measure of the Importance of Effects , 1979 .

[15]  Jum C. Nunnally,et al.  The Place of Statistics in Psychology , 1960 .

[16]  R. Rosenthal,et al.  Assessing the statistical and social importance of the effects of psychotherapy. , 1983, Journal of consulting and clinical psychology.

[17]  W. W. Rozeboom.  The fallacy of the null-hypothesis significance test. , 1960, Psychological bulletin.

[18]  R. Rosenthal,et al.  Mediation of interpersonal expectancy effects: 31 meta-analyses. , 1985 .

[19]  D. Mook,et al.  In defense of external invalidity. , 1983 .

[20]  A. L. Edwards,et al.  An introduction to linear regression and correlation. , 1985 .

[21]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[22]  Judith Spencer Reitman,et al.  Mechanisms of forgetting in short-term memory. , 1971 .

[23]  W. R. Garner,et al.  Operationism and the concept of perception. , 1956, Psychological review.

[24]  J. Reitman.  Without surreptitious rehearsal, information in short-term memory decay , 1974 .

[25]  Ernest Nagel,et al.  An Introduction to Logic and Scientific Method , 1934, Nature.

[26]  A BINDER,et al.  Further considerations on testing the null hypothesis and the strategy and tactics of investigating theoretical models. , 1963, Psychological review.

[27]  D. A. Grant,et al.  Testing the null hypothesis and the strategy and tactics of investigating theoretical models. , 1962, Psychological review.