The recently published study by Rothkopf (1966) illustrates a common, albeit undesirable, phenomenon in behavioral research: The design and analysis are not adequately congruent. Rothkopf's study reflects unusual care and insight into the matter of design and internal validity (with one major deficiency; he does not indicate that the subjects were randomly assigned to treatments, although there is good reason to expect that this was the case). The authors are in complete sympathy with the position that design is of primary importance, and that statistical analysis is secondary, that statistics should be the researcher's slave, not his master. Such a position, however, is not equivalent to, nor an excuse for, selecting inappropriate statistical techniques and models which lead to crude probability estimates of type-I errors, these being, of course, the basis for inference. Rothkopf's study examines the effects of experimental test-like questions (EQs) on a general achievement test (GT) for various reading selections varying the temporal position of EQs [before (B) and after (A)], with and without given answers. He also compared the effects of interspersed EQs versus EQs given in a block prior to reading. He also employed two control groups (no EQs), one of which was given no specialized instructions (C) while the other was directed to read carefully and slowly (called Direction Reference Group DRG by Rothkopf). Rothkopf's (1966) seven-level one-way ANOVA design is illustrated below, showing the combinations of the above-mentioned variables which he included. EQs No EQs (controls)
[1]
David R. Cox,et al.
A Remark on Multiple Comparison Methods
,
1965
.
[2]
W. Hays.
Statistics for psychologists
,
1963
.
[3]
Allen L. Edwards,et al.
Experimental Design in Psychological Research.
,
1951
.
[4]
E. Rothkopf.
Learning from Written Instructive Materials: An Exploration of the Control of Inspection Behavior by Test-Like Events
,
1966
.
[5]
Jack N. Sparks.
Expository Notes On The Problem Of Making Multiple Comparisons In A Completely Randomized Design
,
1963
.
[6]
O. J. Dunn.
Multiple Comparisons among Means
,
1961
.
[7]
M A Walker,et al.
Concepts and Methods of Experimental Statistics
,
1967
.
[8]
L. A. Marascuilo.
Large-sample multiple comparisons.
,
1966,
Psychological bulletin.
[9]
H. Robinson.
Principles and Procedures of Statistics
,
1961
.
[10]
D. B. Duncan,et al.
A Bayesian Approach to Multiple Comparisons
,
1965
.