Auditing Algorithms: Understanding Algorithmic Systems from the Outside In

Algorithms are ubiquitous and critical sources of information online, increasingly acting as gatekeepers for users accessing or sharing information about virtually any topic, including their personal lives and those of friends and family, news and politics, entertainment, and even information about health and well-being. As a result, algorithmically-curated content is drawing increased attention and scrutiny from users, the media, and lawmakers alike. However, studying such content poses considerable challenges, as it is both dynamic and ephemeral: these algorithms are constantly changing, and frequently changing silently, with no record of the content to which users have been exposed over time. One strategy that has proven effective is the algorithm audit: a method of repeatedly querying an algorithm and observing its output in order to draw conclusions about the algorithm’s opaque Danaë Metaxa, Joon Sung Park, Ronald E. Robertson, Karrie Karahalios, Christo Wilson, Jeff Hancock and Christian Sandvig (2021), “Auditing Algorithms”, Foundations and Trends® in Human-Computer Interaction: Vol. 14, No. 4, pp 272–344. DOI: 10.1561/1100000083. ©2021 D. Metaxa et al.

[1]  Michael S. Bernstein,et al.  ESR: Ethics and Society Review of Artificial Intelligence Research , 2021, ArXiv.

[2]  Abubakar Abid,et al.  Large language models associate Muslims with violence , 2021, Nature Machine Intelligence.

[3]  Motahhare Eslami,et al.  Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors , 2021, Proc. ACM Hum. Comput. Interact..

[4]  Brent J. Hecht,et al.  A Deeper Investigation of the Importance of Wikipedia Links to Search Engine Results , 2021, Proc. ACM Hum. Comput. Interact..

[5]  Christo Wilson,et al.  Building and Auditing Fair Algorithms: A Case Study in Candidate Screening , 2021, FAccT.

[6]  Jack Bandy,et al.  Problematic Machine Behavior , 2021, Proc. ACM Hum. Comput. Interact..

[7]  Jatinder Singh,et al.  Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems , 2021, FAccT.

[8]  Tanushree Mitra,et al.  Auditing E-Commerce Platforms for Algorithmically Curated Vaccine Misinformation , 2021, CHI.

[9]  Shea Brown,et al.  The algorithm audit: Scoring the algorithms that score us , 2021, Big Data Soc..

[10]  Adriano Soares Koshiyama,et al.  Towards Algorithm Auditing: A Survey on Managing Legal, Ethical and Technological Risks of AI, ML and Associated Algorithms , 2021, SSRN Electronic Journal.

[11]  Luca Oneto,et al.  Fairness in Machine Learning , 2020, INNSBDDL.

[12]  Tanushree Mitra,et al.  Measuring Misinformation in Video Search Platforms: An Audit Study on YouTube , 2020, Proc. ACM Hum. Comput. Interact..

[13]  Karrie Karahalios,et al.  Auditing Race and Gender Discrimination in Online Housing Markets , 2020, ICWSM.

[14]  Anna Kawakami,et al.  The Media Coverage of the 2020 US Presidential Election Candidates through the Lens of Google's Top Stories , 2020, ICWSM.

[15]  N. Diakopoulos,et al.  Partisan search behavior and Google results in the 2018 U.S. midterm elections , 2020, Information, Communication & Society.

[16]  Aleksandra Urman,et al.  How search engines disseminate information about COVID-19 and why they should do better , 2020, Harvard Kennedy School Misinformation Review.

[17]  Eni Mustafaraj,et al.  The case for voter-centered audits of search engines during political elections , 2020, FAT*.

[18]  Brian W. Powers,et al.  Dissecting racial bias in an algorithm used to manage the health of populations , 2019, Science.

[19]  E. Machery.  What Is a Replication? , 2019, Philosophy of Science.

[20]  Nicholas Diakopoulos,et al.  Auditing News Curation Systems: A Case Study Examining Algorithmic and Editorial Logic in Apple News , 2019, ICWSM.

[21]  David Lazer,et al.  Auditing Autocomplete: Suggestion Networks and Recursive Algorithm Interrogation , 2019, WebSci.

[22]  Brent J. Hecht,et al.  Measuring the Importance of User-Generated Content to Search Engines , 2019, ICWSM.

[23]  M. Prosperi,et al.  Is it time to rethink institutional review boards for the era of big data? , 2019, Nature Machine Intelligence.

[24]  Zubair Shafiq,et al.  Measuring Political Personalization of Google News Search , 2019, WWW.

[25]  Desheng Hu,et al.  Auditing the Partisanship of Google Search Snippets , 2019, WWW.

[26]  Meredith Durbin,et al.  A Mulching Proposal: Analysing and Improving an Algorithmic System for Turning the Elderly into High-Nutrient Slurry , 2019, CHI Extended Abstracts.

[27]  Nicholas Diakopoulos,et al.  Search as News Curator: The Role of Google in Shaping Attention to News Information , 2019, CHI.

[28]  Piotr Sapiezynski,et al.  Quantifying the Impact of User Attentionon Fair Group Representation in Ranked Lists , 2019, WWW.

[29]  Inioluwa Deborah Raji,et al.  Actionable Auditing: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Products , 2019, AIES.

[30]  Ben Hutchinson,et al.  50 Years of Test (Un)fairness: Lessons for Machine Learning , 2018, FAT.

[31]  David Lazer,et al.  Auditing Partisan Audience Bias within Google Search , 2018, Proc. ACM Hum. Comput. Interact..

[32]  Anja Bechmann,et al.  Are We Exposed to the Same “News” in the News Feed? , 2018, Digital Journalism.

[33]  Krishna P. Gummadi,et al.  Search bias quantification: investigating political bias in social media and web search , 2018, Information Retrieval Journal.

[34]  Jennifer Wortman Vaughan,et al.  Using Search Queries to Understand Health Information Needs in Africa , 2018, ICWSM.

[35]  F. Tripodi.  Searching for Alternative Facts , 2018 .

[36]  Eni Mustafaraj,et al.  Investigating the Effects of Google's Search Engine Result Page in Evaluating the Credibility of Online News Sources , 2018, WebSci.

[37]  Christo Wilson,et al.  Investigating the Impact of Gender on Rank in Resume Search Engines , 2018, CHI.

[38]  Anne Marie Piper,et al.  Addressing Age-Related Bias in Sentiment Analysis , 2018, CHI.

[39]  David Lazer,et al.  Auditing the Personalization and Composition of Politically-Related Search Engine Results Pages , 2018, WWW.

[40]  Safiya Noble,et al.  Algorithms of Oppression , 2018 .

[41]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[42]  Krishna P. Gummadi,et al.  Potential for Discrimination in Online Targeted Advertising , 2018, FAT.

[43]  David Lazer,et al.  Suppressing the Search Engine Manipulation Effect (SEME) , 2017, Proc. ACM Hum. Comput. Interact..

[44]  Maria Eriksson,et al.  Tracking Gendered Streams , 2017 .

[45]  Brent J. Hecht,et al.  The Substantial Interdependence of Wikipedia and Google: A Case Study on the Relationship Between Peer Production Communities and Information Technologies , 2017, ICWSM.

[46]  Shilad Sen,et al.  Digital Hegemonies: The Localness of Search Engine Results , 2017 .

[47]  Evaggelia Pitoura,et al.  On Measuring Bias in Online Information , 2017, SGMD.

[48]  Fernando Diaz,et al.  Auditing Search Engines for Differential Satisfaction Across Demographics , 2017, WWW.

[49]  David García,et al.  Bias in Online Freelance Marketplaces: Evidence from TaskRabbit and Fiverr , 2017, CSCW.

[50]  Stefania Milan,et al.  The Alternative Epistemologies of Data Activism , 2016 .

[51]  Michael Luca,et al.  Racial Discrimination in the Sharing Economy: Evidence from a Field Experiment , 2016 .

[52]  Heather Kugelmass.  “Sorry, I’m Not Accepting New Patients” , 2016, Journal of health and social behavior.

[53]  M. Stano,et al.  Insurance, race/ethnicity, and sex in the search for a new physician , 2015 .

[54]  David Lazer,et al.  Location, Location, Location: The Impact of Geolocation on Web Search Personalization , 2015, Internet Measurement Conference.

[55]  Christo Wilson,et al.  Peeking Beneath the Hood of Uber , 2015, Internet Measurement Conference.

[56]  Ronald E. Robertson,et al.  The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections , 2015, Proceedings of the National Academy of Sciences.

[57]  Andrew C. Miller,et al.  Advances in nowcasting influenza-like illness rates using search query logs , 2015, Scientific Reports.

[58]  Andrea Ballatore,et al.  Google chemtrails: A methodology to analyze topic representation in search engine results , 2015, First Monday.

[59]  Sean A. Munson,et al.  Unequal Representation and Gender Stereotypes in Image Search Results for Occupations , 2015, CHI.

[60]  David Lazer,et al.  Measuring Price Discrimination and Steering on E-commerce Web Sites , 2014, Internet Measurement Conference.

[61]  Ryen W. White,et al.  Seeking and sharing health information online: comparing search engines and social media , 2014, CHI.

[62]  Peter Johannes Schulz,et al.  The Impact of Search Engine Selection and Sorting Criteria on Vaccination Beliefs and Attitudes: Two Experiments Manipulating Google Output , 2014, Journal of medical Internet research.

[63]  D. Lazer,et al.  The Parable of Google Flu: Traps in Big Data Analysis , 2014, Science.

[64]  Nick Feamster,et al.  Exposing Inconsistent Web Search Results with Bobble , 2014, PAM.

[65]  Michael Luca,et al.  Digital Discrimination: The Case of Airbnb.com , 2014 .

[66]  S. Noble.  Google Search: Hyper-visibility as a Means of Rendering Black Women and Girls Invisible , 2013, InVisible Culture.

[67]  Balachander Krishnamurthy,et al.  Measuring personalization of web search , 2013, WWW.

[68]  Latanya Sweeney,et al.  Discrimination in online ad delivery , 2013, CACM.

[69]  Vijay Erramilli,et al.  Detecting price and search discrimination on the internet , 2012, HotNets-XI.

[70]  S. Mullainathan,et al.  The Market for Financial Advice: An Audit Study , 2012 .

[71]  Marc-Allen Cartright,et al.  Intentions and attention in exploratory health search , 2011, SIGIR.

[72]  Wendy Hui Kyong Chun,et al.  Programmed Visions: Software and Memory , 2011 .

[73]  Laura A. Granka.  The Politics of Search: A Decade Retrospective , 2010, Inf. Soc..

[74]  Kathy S. Mack.  A forgotten history , 2010 .

[75]  Ryen W. White,et al.  Cyberchondria: Studies of the escalation of medical concerns in Web search , 2009, TOIS.

[76]  Philip Oreopoulos.  Why Do Skilled Immigrants Struggle in the Labor Market? a Field Experiment with Six Thousand Resumes , 2009 .

[77]  Jeremy Ginsberg,et al.  Detecting influenza epidemics using search engine query data , 2009, Nature.

[78]  Michael L. Nelson,et al.  Agreeing to disagree: search engines and their public interfaces , 2007, JCDL '07.

[79]  Thorsten Joachims,et al.  In Google We Trust: Users' Decisions on Rank, Position, and Relevance , 2007, J. Comput. Mediat. Commun..

[80]  Filip Radlinski,et al.  Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search , 2007, TOIS.

[81]  A Vespignani,et al.  Topical interests and the mitigation of search engine bias , 2006, Proceedings of the National Academy of Sciences.

[82]  Eric Goldman,et al.  Search Engine Bias and the Demise of Search Engine Utopianism , 2006 .

[83]  Abbe Mowshowitz,et al.  Measuring search engine bias , 2005, Inf. Process. Manag..

[84]  Abbe Mowshowitz,et al.  Bias on the web , 2002, CACM.

[85]  Helen Nissenbaum,et al.  Shaping the Web: Why the Politics of Search Engines Matters , 2000, Inf. Soc..

[86]  C. Lee Giles,et al.  Accessibility of information on the web , 1999, Nature.

[87]  J. Yinger,et al.  Evidence on Discrimination in Consumer Markets , 1998 .

[88]  Raymond J. Struyk,et al.  Clear and Convincing Evidence: Measurement of Discrimination in America. , 1994 .

[89]  Ronald E. Anderson.  ACM code of ethics and professional conduct , 1992, CACM.

[90]  K. Crenshaw.  Mapping the margins: intersectionality, identity politics, and violence against women of color , 1991 .

[91]  J. Landay,et al.  SearchMedia and Elections : A Longitudinal Investigation of Political Search Results in the 2018 U . S . Elections , 2019 .

[92]  Annabel Rothschild,et al.  How the Interplay of Google and Wikipedia Affects Perceptions of Online News Sources , 2018 .

[93]  Yada Pruksachatkun,et al.  Manipulation of Search Engine Results during the 2016 US Congressional Elections , 2017 .

[94]  Karrie Karahalios,et al.  Auditing Algorithms : Research Methods for Detecting Discrimination on Internet Platforms , 2014 .

[95]  Panagiotis Takis Metaxas,et al.  The Battle for the 2008 US Congressional Elections on the Web , 2009 .

[96]  S. Benard,et al.  Getting a job: Is there a motherhood penalty? , 2005 .

[97]  Abbe Mowshowitz,et al.  Assessing bias in search engines , 2002, Inf. Process. Manag..

[98]  James J. Heckman,et al.  The Urban Institute Audit Studies: Their Methods and Findings , 1993 .

[99]  W. Daniel,et al.  Racial discrimination in England : based on the P.E.P. report , 1968 .