Model Ambiguity in R: The sValues Package

A researcher is studying economic growth and is specifically interested in the role of Government (Nominal) GDP Share. After trying some preliminary models, he comes up with a “good”, “parsimonious” specification with 10 control variables. The coefficient is negative, “significant” and it even resists some “robustness” checks. How reliable is this finding? Actually, not much. But this practice is quite common. Researchers usually engage in ad-hoc specification searches but present only their favorite models. This, however, can easily underestimate the uncertainty caused by model selection and lead to overconfident inferences. Since we are dealing with nonexperimental data, the set of controls can be virtually unlimited and the theory ambiguous about which ones do matter. In this example, it turns out that one can come up with a different set of 10 controls in which the coefficient for Government GDP Share is positive and“significant”. In fact, there are 67 possible control variables, which could generate 148 quintillion different models! So how can we tackle that problem? This presentation will introduce the R package sValues, which implements a measure of sturdiness of coefficients proposed by Leamer[3] and discussed in Leamer[4]. The S-values try to provide a sensible framework to assess the sensitivity of coefficient estimates to model ambiguity. But before going to the R implementation, let’s see a brief description of the method. ∗This vignette is a draft based on a poster presented on useR! 2015. I’ve learned a great deal from discussions with Ed Leamer! I also thank Rasmus Baath, Danilo Freire and Douglas Araujo for their comments. Of course, all remaining errors are my own. And all opinions expressed in this material are mine and do not necessarily reflect the views of the CBB. Contact: carloscinelli@hotmail.com