“Suppose you have run an experiment on 20 subjects, and have obtained a significant result which confirms your theory ( z = 2.23, p If you feel that the probability is somewhere around .85, you may be pleased to know that you belong to a majority group. Indeed, that was the median answer of two small groups who were kind enough to respond to a questionnaire distributed at meetings of the Mathematical Psychology Group and of the American Psychological Association. On the other hand, if you feel that the probability is around .48, you belong to a minority. Only 9 of our 84 respondents gave answers between .40 and .60. However, .48 happens to be a much more reasonable estimate than .85. Apparently, most psychologists have an exaggerated belief in the likelihood of successfully replicating an obtained finding. The sources of such beliefs, and their consequences for the conduct of scientific inquiry, are what this paper is about. Our thesis is that people have strong intuitions about random sampling; that these intuitions are wrong in fundamental respects; that these intuitions are shared by naive subjects and by trained scientists; and that they are applied with unfortunate consequences in the course of scientific inquiry.
[1]
Jacob Cohen,et al.
The statistical power of abnormal-social psychological research: a review.
,
1962,
Journal of abnormal and social psychology.
[2]
Jacob Cohen,et al.
THE STATISTICAL POWER OF ABNORMAL-SOCI AL PSYCHOLOGICAL RESEARCH:
,
1962
.
[3]
A. W. Melton.
Categories of Human Learning
,
1964
.
[4]
G. S. Tune.
RESPONSE PREFERENCES: A REVIEW OF SOME RELEVANT LITERATURE.
,
1964,
Psychological bulletin.
[5]
Jacob Cohen.
Statistical Power Analysis for the Behavioral Sciences
,
1969,
The SAGE Encyclopedia of Research Design.
[6]
John E. Overall.
Classical statistical hypotheses testing within the context of Bayesian theory.
,
1969
.