The Logic of the Survey Experiment Reexamined

Scholars of political behavior increasingly embed experimental designs in opinion surveys by randomly assigning respondents alternative versions of questionnaire items. Such experiments have major advantages: they are simple to implement and they dodge some of the difficulties of making inferences from conventional survey data. But survey experiments are no panacea. We identify problems of inference associated with typical uses of survey experiments in political science and highlight a range of difficulties, some of which have straightforward solutions within the survey-experimental approach and some of which can be dealt with only by exercising greater caution in interpreting findings and bringing to bear alternative strategies of research.

[1]  J. B. Mcconahay,et al.  The scar of race. , 1996 .

[2]  J. Druckman,et al.  Mind, Will, and Choice , 2006 .

[3]  R. Inglehart.  Culture Shift in Advanced Industrial Society , 1991 .

[4]  James N. Druckman,et al.  The Implications of Framing Effects for Citizen Competence , 2001 .

[5]  Diana C. Mutz.  Impersonal Influence: How Perceptions of Mass Collectives Affect Political Attitudes , 1998 .

[6]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme , 1997 .

[7]  R. Lalonde.  Evaluating the Econometric Evaluations of Training Programs with Experimental Data , 1984 .

[8]  Jeffery J. Mondak.  Source Cues and Policy Approval: The Cognitive Dynamics of Public Support for the Reagan Agenda , 1993 .

[9]  H. Schuman,et al.  Survey-Based Experiments on White Racial Attitudes Toward Residential Integration , 1988, American Journal of Sociology.

[10]  D. Davis,et al.  Stereotype Threat and Race of Interviewer Effects in a Survey on Political Knowledge , 2003 .

[11]  Willem E. Saris,et al.  Studies in public opinion : attitudes, nonattitudes, measurement error, and change , 2004 .

[12]  Chris McIntyre,et al.  The Effect of Economic Priorities on the Measurement of Value Change: New Experimental Evidence , 1999, American Political Science Review.

[13]  D. Rucinski.  The Nature and Origins of Mass Opinion. , 1994 .

[14]  Markus Prior,et al.  Predisposing Factors and Situational Triggers: Exclusionary Reactions to Immigrant Minorities , 2004, American Political Science Review.

[15]  Edward G. Carmines,et al.  Reaching Beyond Race , 1997, PS: Political Science & Politics.

[16]  Stanley Lieberson,et al.  Making It Count: The Improvement of Social Research and Theory. , 1987 .

[17]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[18]  Alastair Smith,et al.  Testing theories of strategic choice: The example of crisis escalation , 1999 .

[19]  Leonard A. Steverson.  Rethinking Social Inquiry: Diverse Tools, Shared Standards , 2005 .

[20]  Mathew D. McCubbins,et al.  The Democratic Dilemma: Can Citizens Learn What They Need to Know? , 1998 .

[21]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator , 1998 .

[22]  Paul M. Sniderman,et al.  Innovations in Experimental Design in Attitude Surveys , 1996 .

[23]  Charles S. Taber,et al.  Motivated Skepticism in the Evaluation of Political Beliefs , 2006 .

[24]  Christopher H. Achen.  The Statistical Analysis of Quasi-Experiments , 2023 .

[25]  James H. Kuklinski,et al.  Racial Attitudes and the "New South" , 1997, The Journal of Politics.

[26]  Richard A. Brody,et al.  Reasoning and Choice: Explorations in Political Psychology , 1991 .

[27]  J. Druckman,et al.  Framing and Deliberation: How Citizens' , 2003 .

[28]  Jon Hurwitz,et al.  Racial Stereotypes and Whites' Political Views of Blacks in the Context of Welfare and Crime STOR , 1997 .

[29]  James S. Fishkin,et al.  Considered Opinions: Deliberative Polling in Britain , 2002 .

[30]  William Allen,et al.  The influence of source credibility on communication effectiveness , 1953 .

[31]  Martin Gilens,et al.  Political Ignorance and Collective Policy Preferences , 2001, American Political Science Review.

[32]  T. Rudolph,et al.  Value Conflict, Group Affect, and the Issue of Campaign Finance , 2003 .

[33]  Stephen Ansolabehere,et al.  Going negative : how attack ads shrink and polarize the electorate , 1995 .

[34]  Diana C. Mutz,et al.  The New Videomalaise: Effects of Televised Incivility on Political Trust , 2005, American Political Science Review.

[35]  James D. Fearon,et al.  Counterfactuals and Hypothesis Testing in Political Science , 1991, World Politics.

[36]  D. Kinder,et al.  Divided by Color: Racial Politics and Democratic Ideals , 1996 .

[37]  David Collier,et al.  Rethinking Social Inquiry: Diverse Tools, Shared Standards , 2004 .

[38]  Jon Hurwitz,et al.  Public Perceptions of Race and Crime: The Role of Racial Stereotypes , 1997 .

[39]  J. Gibson.  A Sober Second Thought: An Experiment in Persuading Russians to Tolerate , 1998 .

[40]  Nojin Kwak,et al.  Public Opinion Quarterly , 2022 .

[41]  Thomas E. Nelson,et al.  Media Framing of a Civil Liberties Conflict and Its Effect on Tolerance , 1997, American Political Science Review.

[42]  James J. Heckman,et al.  Characterizing Selection Bias Using Experimental Data , 1998 .

[43]  Dean Lacy,et al.  A Theory of Nonseparable Preferences in Survey Responses , 2001 .

[44]  Robert O. Keohane,et al.  Designing Social Inquiry: Scientific Inference in Qualitative Research. , 1995 .

[45]  James N. Druckman,et al.  Framing and Deliberation: How Citizens' Conversations Limit Elite Influence , 2003 .

[46]  M. Lodge,et al.  The Responsive Voter: Campaign Information and the Dynamics of Candidate Evaluation , 1995, American Political Science Review.

[47]  R. Wyer,et al.  The Cognitive and Affective Bases of Political Tolerance Judgments , 1991 .