Experiments and subject sampling

SUMMARY This paper examines the impact of randomisation and blinding in experiments in which subjects act like investigators by attempting to learn about the effectiveness of the administered treatments. We derive the effects that this type of subject inference has on investigator inference. We show that the conditions under which randomisation and blinding induce unbiased estimation when subjects make treatment inferences are extremely strong and unlikely to hold in most experiments. A test for the presence of such subject sampling in blind experiments is proposed, with empirical results from a set of blind clinical trials indicating the occurrence of subject sampling in about one-third of the trials. In experiments, investigators apply statistical analysis to the outcome data to draw inferences about the effects of treatments. This paper hypothesises that subjects, like investigators, make inferences about treatment efficacy from the outcomes that they experience. For example, in an HIV trial the primary outcome may be the strength of a subject's immune system, as measured by CD4 counts. As the trial progresses, each subject, along with the investigator, learns more about the effectiveness of the assigned treatment each successive period by observing the change in his CD4 count. The likelihood that a subject remains in the trial or complies with the treatment protocol thus increases as the health of the subject improves. We refer to this process of subjects making inferences as subject