Specification Uncertainty and Model Averaging

Theory: Data analysts sometimes report (and more often produce) results from many alternative models with different explanatory variables, functional forms, observations, or exogeneity assumptions. Classical statistical theory is ill-suited to make sense of this practice. Hypotheses: Bayesian statisticians have recently proposed a coherent procedure for taking account of specification uncertainty by averaging results from a variety of different model specifications. The model-averaging procedure has the general effect of discounting evidence derived from elaborate specification searches, especially when alternative models produce markedly different results. Methods: I describe the model-averaging procedure, and illustrate its application using examples drawn from a controversy in comparative political economy between Lange and Garrett (1985, 1987) and Jackman (1987), and from the work of Erikson, Wright, and McIver (1993) on public opinion and policy in the American states. In addition, I propose two classes of reference priors that might usefully supplement the uniform model priors typically adopted in model averaging-a "dummy-resistant prior" for dealing with outlier observations, and a family of "search-resistant priors" for representing sequential specification searches. Results: The model-averaging procedure seems to offer a convenient approximation to full-blown Bayesian analysis in typical social science settings. It is simple to implement, and uses the variety of alternative model specifications already being produced by data analysts to shed some useful light on the inferential implications of specification uncertainty.