Searching for Representation: A sociotechnical audit of googling for members of U.S. Congress

High-quality online civic infrastructure is increasingly critical for the success of democratic processes. There is a pervasive reliance on search engines to find facts and information necessary for political participation and oversight. We find that approximately 10% of the top Google search results are likely to mislead California information seekers who use search to identify their congressional representatives. 70% of the misleading results appear in featured snippets above the organic search results. We use both qualitative and quantitative methods to understand what aspects of the information ecosystem lead to this sociotechnical breakdown. Factors identified include Google’s heavy reliance on Wikipedia, the lack of authoritative, machine parsable, high accuracy data about the identity of elected officials based on geographic location, and the search engine’s treatment of under-specified queries. We recommend steps that Google can take to meet its stated commitment to providing high quality civic information, and steps that information providers can take to improve the legibility and quality of information about congressional representatives available to search algorithms.

[1]  S. Oates,et al.  The Google voter: search engines and elections in the new media ecology , 2018 .

[2]  Philipp Schaer,et al.  An Investigation of Biases in Web Search Engine Query Suggestions , 2019, Online Inf. Rev..

[3]  Eni Mustafaraj,et al.  Investigating the Effects of Google's Search Engine Result Page in Evaluating the Credibility of Online News Sources , 2018, WebSci.

[4]  H. Nissenbaum A Contextual Approach to Privacy Online , 2011, Daedalus.

[5]  D. Boyd,et al.  Data Voids: Where Missing Data Can Easily Be Exploited , 2018 .

[6]  Krishna P. Gummadi,et al.  Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media , 2017, CSCW.

[7]  Nicholas Diakopoulos,et al.  Search as News Curator: The Role of Google in Shaping Attention to News Information , 2019, CHI.

[8]  Davide Buscaldi Toponym ambiguity in geographical information retrieval , 2009, SIGIR.

[9]  S. Edgerly,et al.  Political Inequalities Start at Home: Parents, Children, and the Socialization of Civic Infrastructure Online , 2018 .

[10]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[11]  Brent J. Hecht,et al.  The Substantial Interdependence of Wikipedia and Google: A Case Study on the Relationship Between Peer Production Communities and Information Technologies , 2017, ICWSM.

[12]  David Lazer,et al.  Auditing the Personalization and Composition of Politically-Related Search Engine Results Pages , 2018, WWW.

[13]  David Lazer,et al.  Location, Location, Location: The Impact of Geolocation on Web Search Personalization , 2015, Internet Measurement Conference.

[14]  Tarleton Gillespie,et al.  Algorithmically recognizable: Santorum’s Google problem, and Google’s Santorum problem , 2017, The Social Power of Algorithms.

[15]  Yada Pruksachatkun,et al.  Manipulation of Search Engine Results during the 2016 US Congressional Elections , 2017 .

[16]  I. Somin Knowledge about ignorance: New directions in the study of political information , 2006, The Nature of Belief Systems Reconsidered.

[17]  Betsy Sinclair,et al.  Googling the Top Two: Information Search in California’s Top Two Primary , 2015 .

[18]  Eni Mustafaraj,et al.  The case for voter-centered audits of search engines during political elections , 2020, FAT*.

[19]  Eli Pariser,et al.  The Filter Bubble: What the Internet Is Hiding from You , 2011 .

[20]  David Lazer,et al.  Auditing Partisan Audience Bias within Google Search , 2018, Proc. ACM Hum. Comput. Interact..

[21]  N. Diakopoulos,et al.  Partisan search behavior and Google results in the 2018 U.S. midterm elections , 2020, Information, Communication & Society.

[22]  Brent J. Hecht,et al.  Measuring the Importance of User-Generated Content to Search Engines , 2019, ICWSM.

[23]  Bianca C. Reisdorf,et al.  Social Shaping of the Politics of Internet Search and Networking: Moving Beyond Filter Bubbles, Echo Chambers, and Fake News , 2017 .

[24]  Larry M. Bartels Uninformed Votes: Information E ects in Presidential Elections , 1996 .

[25]  Franziska Pradel Biased Representation of Politicians in Google and Wikipedia Search? The Joint Effect of Party Identity, Gender Identity and Elections , 2020 .

[26]  Desheng Hu,et al.  Auditing the Partisanship of Google Search Snippets , 2019, WWW.

[27]  D. Tambini,et al.  Digital Dominance: The Power of Google, Amazon, Facebook and Apple , 2018 .

[28]  Taina Bucher,et al.  The algorithmic imaginary: exploring the ordinary affects of Facebook algorithms , 2017, The Social Power of Algorithms.

[29]  Danna Zhou,et al.  d. , 1840, Microbial pathogenesis.

[30]  Stefan Geiß,et al.  Seek and you shall find? A content analysis on the diversity of five search engines’ results on political queries , 2020, Information, Communication & Society.

[31]  Inioluwa Deborah Raji,et al.  Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing , 2020, FAT*.

[32]  Brent J. Hecht,et al.  A Deeper Investigation of the Importance of Wikipedia Links to Search Engine Results , 2021, Proc. ACM Hum. Comput. Interact..

[33]  David Lazer,et al.  Auditing Autocomplete: Suggestion Networks and Recursive Algorithm Interrogation , 2019, WebSci.

[34]  Ronald E. Robertson,et al.  The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections , 2015, Proceedings of the National Academy of Sciences.

[35]  Helen Nissenbaum,et al.  Shaping the Web: Why the Politics of Search Engines Matters , 2000, Inf. Soc..

[36]  Brent J. Hecht,et al.  Examining Wikipedia With a Broader Lens: Quantifying the Value of Wikipedia's Relationships with Other Large-Scale Online Communities , 2018, CHI.

[37]  장윤희,et al.  Y. , 2003, Industrial and Labor Relations Terms.

[38]  Tarleton Gillespie,et al.  The politics of ‘platforms’ , 2010, New Media Soc..

[39]  Karrie Karahalios,et al.  Auditing Algorithms : Research Methods for Detecting Discrimination on Internet Platforms , 2014 .