Validation: What Big Data Reveal About Survey Misreporting and the Real Electorate

Social scientists rely on surveys to explain political behavior. From consistent overreporting of voter turnout, it is evident that responses on survey items may be unreliable and lead scholars to incorrectly estimate the correlates of participation. Leveraging developments in technology and improvements in public records, we conduct the first-ever fifty-state vote validation. We parse overreporting due to response bias from overreporting due to inaccurate respondents. We find that nonvoters who are politically engaged and equipped with politically relevant resources consistently misreport that they voted. This finding cannot be explained by faulty registration records, which we measure with new indicators of election administration quality. Respondents are found to misreport only on survey items associated with socially desirable outcomes, which we find by validating items beyond voting, like race and party. We show that studies of representation and participation based on survey reports dramatically misestimate the differences between voters and nonvoters.

[1]  Martyn Denscombe,et al.  Web-Based Questionnaires and the Mode Effect , 2006 .

[2]  Robert F. Belli,et al.  Reducing vote overreporting in surveys : Social desirability, memory failure, and source monitoring , 1999 .

[3]  Michael D. Martinez.  Comment on “Voter Turnout and the National Election Studies” , 2003, Political Analysis.

[4]  Michael J. Hanmer,et al.  Good Excuses: Understanding Who Votes With An Improved Turnout Question , 2007 .

[5]  B. Highton,et al.  The Political Implications of Higher Turnout , 2001, British Journal of Political Science.

[6]  Andrew S. Fullerton,et al.  Bringing Registration into Models of Vote Overreporting , 2007 .

[7]  Henry E. Brady,et al.  Voice and Equality: Civic Voluntarism in American Politics , 1996 .

[8]  Barry C. Burden.  Voter Turnout and the National Election Studies , 2000, Political Analysis.

[9]  J. Citrin,et al.  What if Everyone Voted? Simulating the Impact of Increased Turnout in Senate Elections , 2003 .

[10]  Carol A. Cassel,et al.  Voting Records and Validated Voting Studies , 2004 .

[11]  James E. Campbell.  Explaining Politics, Not Polls Reexamining Macropartisanship with Recalibrated NES Data , 2010 .

[12]  L. Stoker.  The Private Roots of Public Action: Gender, Equality, and Political Participation. By Nancy Burns, Kay Lehman Schlozman, and Sidney Verba. Cambridge, MA: Harvard University Press, 2001. 453p. $55.00 cloth, $27.95 paper , 2003, Perspectives on Politics.

[13]  Matthew N. Beckmann,et al.  What Leads to Voting Overreports? Contrasts of Overreporters to Validated Voters and Admitted Nonvoters in the American National Election Studies , 2001 .

[14]  Michael P. McDonald.  On the Overreport Bias of the National Election Study Turnout Rate , 2003, Political Analysis.

[15]  Michael W. Traugott,et al.  The Consequences of Validated and Self-Reported Voting Measures , 1981 .

[16]  David Brockington,et al.  Social Desirability and Response Validity: A Comparative Analysis of Overreporting Voter Turnout in Five Countries , 2005 .

[17]  R. Belli,et al.  An experimental comparison of question forms used to reduce vote overreporting , 2006 .

[18]  P. Sniderman,et al.  Facing the Challenge of Democracy: Explorations in the Analysis of Public Opinion and Political Participation , 2011 .

[19]  L. Sigelman.  The Nonvoting Voter in Voting Research , 1982 .

[20]  R. Montjoy,et al.  Overreporting voting: why it happens and why it matters. , 2001, Public opinion quarterly.

[21]  Benjamin J. Deufel,et al.  Race And Turnout In U.S. Elections Exposing Hidden Effects , 2010 .

[22]  Paul R. Abramson,et al.  Who Overreports Voting? , 1986, American Political Science Review.

[23]  Barry C. Burden.  Internal and External Effects on the Accuracy of NES Turnout: Reply , 2003, Political Analysis.

[24]  Jonathan N. Katz,et al.  Correcting for Survey Misreports Using Auxiliary Information with an Application to Estimating Turnout , 2008 .

[25]  William T. Harbaugh.  If people vote because they like to, then why do so many of them lie? , 1996 .

[26]  C. A. Cassel.  Overreporting And Electoral Participation Research , 2003 .

[27]  Steven J. Rosenstone,et al.  Mobilization, participation, and democracy in America , 1993 .

[28]  Jon A. Krosnick,et al.  The Effect of Survey Mode and Sampling on Inferences about Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys with Nonprobability Samples , 2007, Political Analysis.