Why Bother with Experiments

Abstract : A generalization is a working hypothesis, typically expressed in the form of cause-effect relations. In the social sciences, generalizations decay because (a) it is difficult to identify appropriate cause-effect relations, and (b) such relations are sensitive to the influences of environmental conditions. Whereas scientists should be realistic in thier aspirations to create generalizable knowledge, much can be done to improve performance through the use of formal models and experimentation. It is particularly important that theories permit comparisons between models and data at multiple levels involving processes, environmental conditions, and predictions. Scientists should avoid and the extremes of 'models without data' and 'data without models.' Instead, models would be subjected to 'strong' empirical tests via predictions (rather than tests of statistical significance), and the competing predictions of alternatives. In addition to suggesting what experimental evidence should be collected, models also serve the important function of determining when data collection would be of little value. The nature of experimental evidence is considered from three viewpoints: (1) asymmetries in the way data and models interact in affecting conclusions; (2) apparent but illusory conflicts between the goals of internal and external validity; and (3) the importance of conducting experiments despite poor prospects of creating knowledge that can be generalized.