Quantifying Gender Biases Towards Politicians on Reddit

Despite attempts to increase gender parity in politics, global efforts have struggled to ensure equal female representation. This is likely tied to implicit gender biases against women in authority. In this work, we present a comprehensive study of gender biases that appear in online political discussion. To this end, we collect 10 million comments on Reddit in conversations about male and female politicians, which enables an exhaustive study of automatic gender bias detection. We address not only misogynistic language, but also benevolent sexism in the form of seemingly positive attitudes examining both sentiment and dominance attributed to female politicians. Finally, we conduct a multi-faceted study of gender bias towards politicians investigating both linguistic and extra-linguistic cues. We assess 5 different types of gender bias, evaluating coverage, combinatorial, nominal, sentimental and lexical biases extant in social media language and discourse. Overall, we find that, contrary to previous research, coverage and sentiment biases suggest equal public interest in female politicians. However, the results of the nominal and lexical analyses suggest this interest is not as professional or respectful as that expressed about male politicians. Female politicians are often named by their first names and are described in relation to their body, clothing, or family; this is a treatment that is not similarly extended to men. On the now banned far-right subreddits, this disparity is greatest, though differences in gender biases still appear in the right and left-leaning subreddits. We release the curated dataset to the public for future studies.

[1]  G. A. Barnard,et al.  Transmission of Information: A Statistical Theory of Communications. , 1961 .

[2]  Sonja Schmer-Galunder,et al.  Relating Word Embedding Gender Biases to Gender Gaps: A Cross-Cultural Analysis , 2019, Proceedings of the First Workshop on Gender Bias in Natural Language Processing.

[3]  S. Lemon,et al.  The Ambivalent Sexism Inventory : Differentiating Hostile and Benevolent Sexism , 2001 .

[4]  Yulia Tsvetkov,et al.  A Framework for the Computational Linguistic Analysis of Dehumanization , 2020, Frontiers in Artificial Intelligence.

[5]  Thanassis Tiropanis,et al.  The problem of identifying misogynist language on Twitter (and other online social spaces) , 2016, WebSci.

[6]  Kathleen Dolan,et al.  The Impact of Gender Stereotyped Evaluations on Support for Women Candidates , 2010 .

[7]  Yulia Tsvetkov,et al.  RtGender: A Corpus for Studying Differential Responses to Gender , 2018, LREC.

[8]  Saif M. Mohammad,et al.  PoKi: A Large Dataset of Poems by Children , 2020, LREC.

[9]  Khalid Choukri,et al.  The european language resources association , 1998, LREC.

[10]  Yejin Choi,et al.  Social Bias Frames: Reasoning about Social and Power Implications of Language , 2020, ACL.

[11]  M. Ferguson,et al.  How gender determines the way we speak about professionals , 2018, Proceedings of the National Academy of Sciences.

[12]  Eduardo Graells-Garrido,et al.  Women through the glass ceiling: gender asymmetries in Wikipedia , 2016, EPJ Data Science.

[13]  K. Pearson On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables is Such that it Can be Reasonably Supposed to have Arisen from Random Sampling , 1900 .

[14]  L. Huddy,et al.  Gender Stereotypes and the Perception of Male and Female Candidates , 1993 .

[15]  Paolo Rosso,et al.  Automatic Identification and Classification of Misogynistic Language on Twitter , 2018, NLDB.

[16]  Jesse Holcomb,et al.  Twitter Makes It Worse: Political Journalists, Gendered Echo Chambers, and the Amplification of Gender Bias , 2018, The International Journal of Press/Politics.

[17]  Michael S. Bernstein,et al.  Shirtless and Dangerous: Quantifying Linguistic Signals of Gender Bias in an Online Fiction Writing Community , 2016, ICWSM.

[18]  Jacob Eisenstein,et al.  You Can't Stay Here , 2017, Proc. ACM Hum. Comput. Interact..

[19]  Eran Shor,et al.  A Large-Scale Test of Gender Bias in the Media , 2019, Sociological Science.

[20]  H. Blossfeld,et al.  Women’s disadvantage in holding supervisory positions. Variations among European countries and the role of horizontal gender segregation , 2017 .

[21]  J. Tukey Comparing individual means in the analysis of variance. , 1949, Biometrics.

[22]  Jason Weston,et al.  Multi-Dimensional Gender Bias Classification , 2020, EMNLP.

[23]  John H. Parmelee,et al.  Gender and Generational Differences in Political Reporters’ Interactivity on Twitter , 2017 .

[24]  Cristian Danescu-Niculescu-Mizil,et al.  Tie-breaker: Using language models to quantify gender bias in sports journalism , 2016, ArXiv.

[25]  Mark Heitmann,et al.  More than a Feeling: Benchmarks for Sentiment Analysis Accuracy , 2020, SSRN Electronic Journal.

[26]  J. Lever,et al.  Does gender bias against female leaders persist? Quantitative and qualitative data from a large-scale survey , 2011 .

[27]  Saif Mohammad,et al.  Obtaining Reliable Human Ratings of Valence, Arousal, and Dominance for 20,000 English Words , 2018, ACL.

[28]  Brian A. Nosek,et al.  Implicit social cognition: from measures to mechanisms , 2011, Trends in Cognitive Sciences.

[29]  F. Massey The Kolmogorov-Smirnov Test for Goodness of Fit , 1951 .

[30]  David García,et al.  It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopedia , 2015, ICWSM.

[31]  Emily Ahn,et al.  Finding Microaggressions in the Wild: A Case for Locating Elusive Phenomena in Social Media Posts , 2019, EMNLP.

[32]  Dan Jurafsky,et al.  Content Analysis of Textbooks via Natural Language Processing: Findings on Gender, Race, and Ethnicity in Texas U.S. History Textbooks , 2020, AERA Open.

[33]  Om P. Damani,et al.  Improving Pointwise Mutual Information (PMI) by Incorporating Significant Co-occurrence , 2013, CoNLL.

[34]  Erik Olin Wright,et al.  The Gender Gap in Workplace Authority: A Cross-National Study. , 1995 .

[35]  Laurie A. Rudman,et al.  Implicit and Explicit Attitudes Toward Female Authority , 2000 .

[36]  Seth Ovadia,et al.  The Glass Ceiling Effect , 2001 .

[37]  Daniel Jurafsky,et al.  Word embeddings quantify 100 years of gender and ethnic stereotypes , 2017, Proceedings of the National Academy of Sciences.

[38]  S. Garikipati,et al.  Leading the Fight Against the Pandemic: Does Gender Really Matter? , 2020, Feminist Economics.

[39]  Fiona M. Kay,et al.  Gender in Practice: A Study of Lawyers' Lives , 1995 .

[40]  A. Greenwald,et al.  Measuring individual differences in implicit cognition: the implicit association test. , 1998, Journal of personality and social psychology.

[41]  Edgar Altszyler,et al.  On the interpretation and significance of bias metrics in texts: a PMI-based approach , 2021, ArXiv.

[42]  Austin R. Benson,et al.  Higher-order Homophily is Combinatorially Impossible , 2021, ArXiv.

[43]  Yulia Tsvetkov,et al.  Unsupervised Discovery of Implicit Gender Bias , 2020, EMNLP.

[44]  Christopher Potts,et al.  TalkDown: A Corpus for Condescension Detection in Context , 2019, EMNLP.

[45]  Diana Adler,et al.  Using Multivariate Statistics , 2016 .

[46]  Yulia Tsvetkov,et al.  Controlled Analyses of Social Biases in Wikipedia Bios , 2021, ArXiv.

[47]  Laurel Smith‐Doerr,et al.  Women, Gender, and Technology , 2007 .

[48]  Ryan Cotterell,et al.  Unsupervised Discovery of Gendered Language through Latent-Variable Modeling , 2019, ACL.

[49]  Yejin Choi,et al.  Connotation Frames of Power and Agency in Modern Films , 2017, EMNLP.

[50]  Viviana Patti,et al.  Misogyny Detection in Twitter: a Multilingual and Cross-Domain Study , 2020, Inf. Process. Manag..

[51]  Chandler May,et al.  Social Bias in Elicited Natural Language Inferences , 2017, EthNLP@EACL.

[52]  Armin Mertens,et al.  As the Tweet, so the Reply?: Gender Bias in Digital Communication with Politicians , 2019, WebSci.

[53]  Casey Fiesler,et al.  Reddit Rules! Characterizing an Ecosystem of Governance , 2018, ICWSM.

[54]  Mounia Lalmas,et al.  First Women, Second Sex: Gender Bias in Wikipedia , 2015, HT.

[55]  Adam Tauman Kalai,et al.  Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings , 2016, NIPS.

[56]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[57]  Olle Folke,et al.  The Glass Ceiling in Politics: Formalization and Empirical Tests , 2014 .

[58]  Harith Alani,et al.  Exploring Misogyny across the Manosphere in Reddit , 2019, WebSci.

[59]  Isabelle Augenstein,et al.  Quantifying gender bias towards politicians in cross-lingual language models , 2021, PloS one.

[60]  Marcel Urner,et al.  Using Multivariate Statistics 5th Edition , 2016 .

[61]  My Nguyen Women Representation in The Media: Gender Bias and Status Implications , 2020 .