Content moderation: Social media’s sexist assemblages

This article proposes ‘sexist assemblages’ as a way of understanding how the human and mechanical elements that make up social media content moderation assemble to perpetuate normative gender roles, particularly white femininities, and to police content related to women and their bodies. It investigates sexist assemblages through three of many potential elements: (1) the normatively gendered content presented to users through in-platform keyword and hashtag searches; (2) social media platforms’ community guidelines, which lay out platforms’ codes of conduct and reveal biases and subjectivities and (3) the over-simplification of gender identities that is necessary to algorithmically recommend content to users as they move through platforms. By the time the reader finds this article, the elements of the assemblages we identify might have shifted, but we hope the framework remains useful for those aiming to understand the relationship between content moderation and long-standing forms of inequality.

[1]  Sarah T. Roberts Content Moderation , 2022, Encyclopedia of Big Data.

[2]  N. K. Hayles Unthought , 2021, Uncertain Archives.

[3]  Anthony McCosker,et al.  Hashtagging depression on Instagram: Towards a more inclusive mental health research methodology , 2020, New Media Soc..

[4]  S. Noble Algorithms of Oppression: How Search Engines Reinforce Racism , 2018 .

[5]  Ysabel Gerrard,et al.  Behind the Screen: Content Moderation in the Shadows of Social Media , 2019, New Media Soc..

[6]  Alison Hearn Verified: Self-presentation, identity management, and selfhood in the age of big data , 2017, Self-(re)presentation now.

[7]  Sarah T. Roberts,et al.  Behind the Screen , 2019 .

[8]  Nicolas Suzor Lawless , 2019 .

[9]  Alisha Karabinus Book Review: Post, Mine, Repeat: Social Media Data Mining Becomes Ordinary , 2019, Convergence: The International Journal of Research into New Media Technologies.

[10]  Tim Squirrell,et al.  Platform dialectics: The relationships between volunteer moderators and end users on reddit , 2019, New Media Soc..

[11]  Elinor Carmi Sonic Publics| The Hidden Listeners: Regulating the Line from Telephone Operators to Content Moderators , 2019 .

[12]  Geoff Kaufman,et al.  Moderator engagement and community development in the age of algorithms , 2019, New Media Soc..

[13]  Jonathan Cohn,et al.  Book Review: We Are Data: Algorithms and the Making of Our Digital Selves , 2018, New Media Soc..

[14]  Mehmet Fatih Çömlekçi Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions that Shape Social Media , 2019 .

[15]  Nicolas Suzor,et al.  Lawless: the secret rules that govern our digital lives , 2018 .

[16]  H. Thornham Gender and Digital Culture , 2018 .

[17]  Taina Bucher If...Then , 2018, Oxford Scholarship Online.

[18]  J. Burgess,et al.  Queer women’s experiences of patchwork platform governance on Tinder, Instagram, and Vine , 2018, Convergence: The International Journal of Research into New Media Technologies.

[19]  Ysabel Gerrard,et al.  Beyond the hashtag: Circumventing content moderation on social media , 2018, New Media Soc..

[20]  Sarah Myers West,et al.  Censored, suspended, shadowbanned: User interpretations of content moderation on social media platforms , 2018, New Media Soc..

[21]  Debbie Ging,et al.  ‘Written in these scars are the stories I can’t explain’: A content analysis of pro-ana and thinspiration image sharing on Instagram , 2018, New Media Soc..

[22]  Ben Light,et al.  The walkthrough method: An approach to the study of apps , 2018, New Media Soc..

[23]  C. Lo,et al.  When all you have is a banhammer : the social and communicative work of Volunteer moderators , 2018 .

[24]  John Cheney-Lippold John Cheney-Lippold, We Are Data: Algorithms and the Making of Our Digital Selves , 2018 .

[25]  Sophia Melanson,et al.  We are data: algorithms and the making of our digital selves , 2017 .

[26]  Ysabel Gerrard Post, Mine, Repeat: social media data mining becomes ordinary , 2017 .

[27]  Gemma Cobb,et al.  “This is not pro-ana”: Denial and disguise in pro-anorexia online spaces , 2017 .

[28]  N. Katherine Hayles,et al.  Unthought: The Power of the Cognitive Nonconscious , 2017 .

[29]  Oliver L. Haimson,et al.  Baking Gender Into Social Media Design: How Platforms Shape Categories for Users and Advertisers , 2016 .

[30]  Anna Lauren Hoffmann,et al.  Constructing and enforcing "authentic" identity online: Facebook, real names, and non-normative identities , 2016, First Monday.

[31]  H. Kennedy Post, Mine, Repeat: Social Media Data Mining Becomes Ordinary , 2016 .

[32]  Safiya Umoja Noble,et al.  The Intersectional Internet: Race, Sex, Class, and Culture Online , 2016 .

[33]  Peter Kelly,et al.  After (a) Method , 2016 .

[34]  Kate Crawford,et al.  What is a flag for? Social media reporting tools and the vocabulary of complaint , 2016, New Media Soc..

[35]  Munmun De Choudhury,et al.  #thyghgapp: Instagram Content Moderation and Lexical Variation in Pro-Eating Disorder Communities , 2016, CSCW.

[36]  Lucas D. Introna Algorithms, Governance, and Governmentality , 2016 .

[37]  H. Kennedy Post, Mine, Repeat , 2016 .

[38]  Celia Lury,et al.  Biosensing: Tracking Persons. , 2016 .

[39]  Sarah T. Roberts Commercial Content Moderation: Digital Laborers' Dirty Work , 2016 .

[40]  David Stuart,et al.  The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences , 2015, Online Inf. Rev..

[41]  Tarleton Gillespie,et al.  Platforms Intervene , 2015 .

[42]  Jordan Frith,et al.  Anonymity, pseudonymity, and the agency of online identity: Examining the social practices of r/Gonewild , 2015, First Monday.

[43]  Rosalind Gill,et al.  Unspeakable Inequalities: Post Feminism, Entrepreneurial Subjectivity, and the Repudiation of Sexism among Cultural Workers , 2014 .

[44]  Rob Kitchin,et al.  The data revolution : big data, open data, data infrastructures & their consequences , 2014 .

[45]  F. Attwood Mainstreaming Sex: The Sexualization of Western Culture , 2014 .

[46]  James Mussell Raw Data is an Oxymoron , 2014 .

[47]  Jeremy Packer,et al.  Subjects, networks, assemblages: a materialist approach to the production of social space , 2013 .

[48]  Pablo J. Boczkowski,et al.  The Relevance of Algorithms , 2013 .

[49]  Genna R. Miller “Gender Trouble” , 2012 .

[50]  Michel Foucoult Discipline and Punish , 2012 .

[51]  Eli Pariser,et al.  The Filter Bubble: What the Internet Is Hiding from You , 2011 .

[52]  A. Saguy,et al.  Morality and health: News media constructions of overweight and eating disorders , 2010 .

[53]  Susan J. Douglas,et al.  Enlightened Sexism: The Seductive Message That Feminism's Work Is Done , 2010 .

[54]  Tarleton Gillespie,et al.  The politics of ‘platforms’ , 2010, New Media Soc..

[55]  Avi Shankar,et al.  Technologies of Sexiness: Theorizing Women’s Engagement in the Sexualization of Culture , 2010 .

[56]  Rosalind Gill Beyond the `Sexualization of Culture' Thesis: An Intersectional Analysis of `Sixpacks',`Midriffs' and `Hot Lesbians' in Advertising , 2009 .

[57]  Rosalind Gill,et al.  Postfeminist media culture , 2007 .

[58]  Susanne Bauer,et al.  Cutting Code. Software and Sociality , 2007 .

[59]  Lucy Suchman,et al.  Human-Machine Reconfigurations: Plans and Situated Actions , 2006 .

[60]  Rupa Huq Beyond Subculture: Pop, Youth and Identity in a Postcolonial World , 2006 .

[61]  J. Law After Method: Mess in Social Science Research , 2004 .

[62]  Karen Dias,et al.  The Ana Sanctuary: Women’s Pro-Anorexia Narratives in Cyberspace , 2003 .

[63]  N. Weatherill,et al.  Introduction * , 1947, Nordic Journal of Linguistics.

[64]  S. Turkle Life on the Screen: Identity in the Age of the Internet , 1997 .

[65]  Susan Bordo Unbearable Weight: Feminism, Western Culture, and the Body , 1993 .

[66]  K. Crenshaw Mapping the margins: intersectionality, identity politics, and violence against women of color , 1991 .

[67]  J. Wajcman Feminism Confronts Technology , 1991 .

[68]  B. Latour Technology is Society Made Durable , 1990 .

[69]  M. P. Root Disordered eating in women of color , 1990 .

[70]  K. Crenshaw Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics , 1989 .