Ch. 4: Statistical research designs for causal inference

In Chapter 3 we have discussed the different ways in which the social sciences conceptualize causation and we have argued that there is no single way in which causal relationships can be defined and analyzed empirically. In this chapter, we focus on a specific set of approaches to constructing research designs for causal analysis, namely, one based on the potential outcomes framework developed in statistics. As discussed in Chapter 3, this perspective is both probabilistic and counterfactual. It is probabilistic because it does not assume that the presence of a given cause leads invariably to a given effect, while it is counterfactual because it involves the comparison of actual configurations with hypothetical alternatives that are not observed in reality. In essence, this approach underscores the necessity to rely on comparable groups in order to achieve valid causal inferences. An important implication is that the design of a study is of paramount importance. The way in which the data are produced is the critical step of the research, while the actual data analysis, while obviously important, plays a secondary role. However, a convincing design requires research questions to be broken down to manageable pieces. Thus, the big tradeoff in this perspective is between reliable inferences on very specific causal relationships on the one hand, and their broader context and complexity (and, possibly, theoretical relevance) on the other hand. The chapter first distinguishes between two general perspectives on causality, namely, one that puts the causes of effects in the foreground, and another that is more interested in the effects of causes. We will then introduce the potential outcomes framework before discussing ∗Chapter for Maggetti, Martino, Fabrizio Gilardi and Claudio M. Radaelli (2012), Research Design in the Social Sciences, SAGE Publications. †Associate Professor, Department of Political Science, University of Zurich, Switzerland. Email: gilardi@ipz.uzh.ch; URL: http://www.fabriziogilardi.org/.

[1]  K. Linos Diffusion through Democracy , 2011 .

[2]  Josef Zweimüller,et al.  How Does Parental Leave Affect Fertility and Return to Work? Evidence from Two Natural Experiments , 2009 .

[3]  Susan D. Hyde THE OBSERVER EFFECT IN INTERNATIONAL POLITICS: Evidence from a Natural Experiment , 2007 .

[4]  N. D. Pidgen,et al.  The Comparative Method , 1987 .

[5]  Rebecca B. Morton,et al.  Experimentation in Political Science , 2008 .

[6]  Donald P. Green,et al.  Testing the Accuracy of Regression Discontinuity Analysis Using Experimental Benchmarks , 2009, Political Analysis.

[7]  James H. Kuklinski,et al.  The Growth and Development of Experimental Research in Political Science , 2006, American Political Science Review.

[8]  Gary King,et al.  Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference , 2007, Political Analysis.

[9]  A. Gelman Causality and Statistical Learning1 , 2010, American Journal of Sociology.

[10]  H. L. Kern,et al.  Opium for the Masses: How Foreign Free Media Can Stabilize Authoritarian Regimes , 2007 .

[11]  T. Dunning,et al.  Cross-cutting Cleavages and Ethnic Voting: An Experimental Study of Cousinage in Mali , 2010, American Political Science Review.

[12]  B. Olken Direct Democracy and Local Public Goods: Evidence from a Field Experiment in Indonesia , 2008, American Political Science Review.

[13]  Alice J. Kang,et al.  The Global Impact of Quotas , 2008 .

[14]  D. Green,et al.  Field Experiments and Natural Experiments , 2008 .

[15]  J. Weinstein,et al.  Field Experiments and the Political Economy of Development , 2009 .

[16]  Gary Goertz,et al.  A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research , 2006, Political Analysis.

[17]  Jens Hainmueller,et al.  MPs for Sale? Returns to Office in Postwar British Politics , 2009, American Political Science Review.

[18]  Joshua D. Angrist,et al.  The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con Out of Econometrics , 2010, SSRN Electronic Journal.

[19]  Shelley J. Correll,et al.  Constraints into Preferences: Gender, Status, and Emerging Career Aspirations , 2004 .

[20]  Jennifer Jerit,et al.  Are Survey Experiments Externally Valid? , 2010, American Political Science Review.

[21]  C. Glymour,et al.  STATISTICS AND CAUSAL INFERENCE , 1985 .

[22]  Jens Hainmueller,et al.  Attitudes toward Highly Skilled and Low-skilled Immigration: Evidence from a Survey Experiment , 2010, American Political Science Review.

[23]  Brad T. Gomez,et al.  Estimating the Electoral Effects of Voter Turnout , 2010, American Political Science Review.

[24]  Rikhil R Bhavnani,et al.  Do Electoral Quotas 'Work' After They Are Withdrawn? Evidence from a Natural Experiment in India , 2008 .

[25]  Kenneth C. Williams,et al.  Experimental Political Science and the Study of Causality: From Nature to the Lab , 2010 .

[26]  Brian J. Gaines,et al.  The Logic of the Survey Experiment Reexamined , 2007, Political Analysis.

[27]  Christopher Winship,et al.  Counterfactuals and Causal Inference by Stephen L. Morgan , 2014 .

[28]  Donald P. Green,et al.  Instrumental Variables Estimation in Political Science: A Readers’ Guide , 2011 .

[29]  James G. Gimpel,et al.  How Large and Long-lasting Are the Persuasive Effects of Televised Campaign Ads? Results from a Randomized Field Experiment , 2011, American Political Science Review.

[30]  Thad Dunning,et al.  Improving Causal Inference , 2008 .

[31]  Christopher Winship,et al.  Counterfactuals and Causal Inference: Methods and Principles for Social Research , 2007 .

[32]  Elisabeth R. Gerber,et al.  When Mayors Matter : Estimating the Impact of Mayoral Partisanship on City Policy , 2011 .

[33]  J. Kmenta Mostly Harmless Econometrics: An Empiricist's Companion , 2010 .

[34]  James A. Robinson,et al.  The Colonial Origins of Comparative Development: An Empirical Investigation , 2000 .