Addressing Age-Related Bias in Sentiment Analysis

Computational approaches to text analysis are useful in understanding aspects of online interaction, such as opinions and subjectivity in text. Yet, recent studies have identified various forms of bias in language-based models, raising concerns about the risk of propagating social biases against certain groups based on sociodemographic factors (e.g., gender, race, geography). In this study, we contribute a systematic examination of the application of language models to study discourse on aging. We analyze the treatment of age-related terms across 15 sentiment analysis models and 10 widely-used GloVe word embeddings and attempt to alleviate bias through a method of processing model training data. Our results demonstrate that significant age bias is encoded in the outputs of many sentiment analysis algorithms and word embeddings. We discuss the models' characteristics in relation to output bias and how these models might be best incorporated into research.

[1]  Christo Wilson,et al.  Peeking Beneath the Hood of Uber , 2015, Internet Measurement Conference.

[2]  TaboadaMaite,et al.  Lexicon-based methods for sentiment analysis , 2011 .

[3]  Eric Gilbert,et al.  VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text , 2014, ICWSM.

[4]  Daniel Gooch,et al.  Communications of the ACM , 2011, XRDS.

[5]  Brent J. Hecht,et al.  Turkers, Scholars, "Arafat" and "Peace": Cultural Communities and Algorithmic Gold Standards , 2015, CSCW.

[6]  Andrew Y. Ng,et al.  Improving Word Representations via Global Context and Multiple Word Prototypes , 2012, ACL.

[7]  Graeme Hirst,et al.  Evaluating WordNet-based Measures of Lexical Semantic Relatedness , 2006, CL.

[8]  Navneet Kaur,et al.  Opinion mining and sentiment analysis , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[9]  Adam Tauman Kalai,et al.  Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings , 2016, NIPS.

[10]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[11]  Adam Tauman Kalai,et al.  Quantifying and Reducing Stereotypes in Word Embeddings , 2016, ArXiv.

[12]  Isaac Record,et al.  Responsible epistemic technologies: A social-epistemological analysis of autocompleted web search , 2017, New Media Soc..

[13]  Fabrício Benevenuto,et al.  A Benchmark Comparison of State-of-the-Practice Sentiment Analysis Methods , 2015, ArXiv.

[14]  A. Greenwald,et al.  Using the implicit association test to measure age differences in implicit social cognitions. , 2002, Psychology and aging.

[15]  Alexei A. Efros,et al.  Unbiased look at dataset bias , 2011, CVPR 2011.

[16]  James Jerger,et al.  Research in aging. , 2009, Journal of the American Academy of Audiology.

[17]  Krishna P. Gummadi,et al.  Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media , 2017, CSCW.

[18]  Anna L. Cox,et al.  Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems , 2019, CHI.

[19]  K. Crenshaw Mapping the margins: intersectionality, identity politics, and violence against women of color , 1991 .

[20]  Philip J. Guo Older Adults Learning Computer Programming: Motivations, Frustrations, and Design Opportunities , 2017, CHI.

[21]  Jeremy P. Birnholtz,et al.  "Algorithms ruin everything": #RIPTwitter, Folk Theories, and Resistance to Algorithmic Change in Social Media , 2017, CHI.

[22]  Shion Guha,et al.  When Subjects Interpret the Data: Social Media Non-use as a Case for Adapting the Delphi Method to CSCW , 2017, CSCW.

[23]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[24]  Mark Davies The Corpus of Contemporary American English (COCA) , 2012 .

[25]  Thomas N. Smyth,et al.  Anti-oppressive design , 2014, Interactions.

[26]  Wai-Tat Fu,et al.  #Snowden: Understanding Biases Introduced by Behavioral Differences of Opinion Groups on Social Media , 2016, CHI.

[27]  A. Greenwald,et al.  Measuring individual differences in implicit cognition: the implicit association test. , 1998, Journal of personality and social psychology.

[28]  Lucas D. Introna,et al.  Picturing Algorithmic Surveillance: The Politics of Facial Recognition Systems , 2002, Surveillance & Society.

[29]  Helen Nissenbaum,et al.  Bias in computer systems , 1996, TOIS.

[30]  Anne Marie Piper,et al.  “Why would anybody do this?”: Older Adults’ Understanding of and Experiences with Crowd Work , 2016 .

[31]  Leysia Palen,et al.  (How) will the revolution be retweeted?: information diffusion and the 2011 Egyptian uprising , 2012, CSCW.

[32]  Helen Nissenbaum,et al.  Defining the Web: The Politics of Search Engines , 2000, Computer.

[33]  Katie Brittain,et al.  An Age-Old Problem: Examining the Discourses of Ageing in HCI and Strategies for Future Research , 2015, TCHI.

[34]  Eduardo Graells-Garrido,et al.  Women through the glass ceiling: gender asymmetries in Wikipedia , 2016, EPJ Data Science.

[35]  原田 秀逸 私の computer 環境 , 1998 .

[36]  Aaron C. T. Smith Older adults and technology use , 2014 .

[37]  Nicholas Diakopoulos,et al.  Algorithmic Accountability , 2015 .

[38]  Latanya Sweeney,et al.  Discrimination in online ad delivery , 2013, CACM.

[39]  Helen Nissenbaum,et al.  How computer systems embody values , 2001, Computer.

[40]  Geraldine Fitzpatrick,et al.  YouTube and intergenerational communication: the case of Geriatric1927 , 2009, Universal Access in the Information Society.

[41]  Frank A. Pasquale The Black Box Society: The Secret Algorithms That Control Money and Information , 2015 .

[42]  Johannes Schöning,et al.  The Effect of Population and "Structural" Biases on Social Media-based Algorithms: A Case Study in Geolocation Inference Across the Urban-Rural Spectrum , 2017, CHI.

[43]  Shaowen Bardzell,et al.  Feminist HCI: taking stock and outlining an agenda for design , 2010, CHI.

[44]  Michael S. Bernstein,et al.  Quantifying the invisible audience in social networks , 2013, CHI.

[45]  Kate Crawford,et al.  Can an Algorithm be Agonistic? Ten Scenes from Life in Calculated Publics , 2016 .

[46]  Ronen Feldman,et al.  Techniques and applications for sentiment analysis , 2013, CACM.

[47]  Lynn Dombrowski,et al.  Social Justice-Oriented Interaction Design: Outlining Key Design Strategies and Commitments , 2016, Conference on Designing Interactive Systems.

[48]  Michael A. DeVito,et al.  From Editors to Algorithms , 2017 .

[49]  Aaron D. Shaw,et al.  Black Lives Matter in Wikipedia: Collective Memory and Collaboration around Online Social Movements , 2016, CSCW.

[50]  Jennifer Ann Rode,et al.  A theoretical agenda for feminist HCI , 2011, Interact. Comput..

[51]  William N. Venables,et al.  Modern Applied Statistics with S , 2010 .

[52]  Christopher Potts,et al.  Learning Word Vectors for Sentiment Analysis , 2011, ACL.

[53]  Paul Baker,et al.  ‘Why do white people have thin lips?’ Google and the perpetuation of stereotypes via auto-complete search forms , 2013 .

[54]  Joanna Bryson,et al.  Semantics derived automatically from language corpora contain human-like biases , 2016, Science.

[55]  Nick Cercone,et al.  Computational Linguistics , 1986, Communications in Computer and Information Science.

[56]  J. Beard,et al.  Valuing older people: time for a global campaign to combat ageism , 2016, Bulletin of the World Health Organization.

[57]  Marlon Twyman Black Lives Matter in Wikipedia: Collaboration and Collective Memory around Online Social Movements , 2016 .

[58]  Ratul Mahajan,et al.  Proceedings of the 2015 Internet Measurement Conference , 2012, IMC 2012.

[59]  Deniz S. Ones,et al.  In hiring, Algorithms beat instinct , 2014 .

[60]  Sofia Ranchordás The Black Box Society: The Secret Algorithms That Control Money and Information by Pasquale Frank Cambridge, MA: Harvard University Press, 2015 , 2016, European Journal of Risk Regulation.

[61]  Anne Marie Piper,et al.  "Tell It Like It Really Is": A Case of Online Content Creation and Sharing Among Older Adult Bloggers , 2016, CHI.

[62]  Ari Rappoport,et al.  Enhanced Sentiment Learning Using Twitter Hashtags and Smileys , 2010, COLING.

[63]  Paul Dourish,et al.  Postcolonial computing: a lens on design and development , 2010, CHI.

[64]  Shaowen Bardzell,et al.  Towards a feminist HCI methodology: social science, feminism, and HCI , 2011, CHI.

[65]  K P Lasher,et al.  Measurement of Aging Anxiety: Development of the Anxiety about Aging Scale , 1993, International journal of aging & human development.

[66]  Sean A. Munson,et al.  Unequal Representation and Gender Stereotypes in Image Search Results for Occupations , 2015, CHI.

[67]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[68]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[69]  Claire Cardie,et al.  OpinionFinder: A System for Subjectivity Analysis , 2005, HLT.

[70]  R. Butler,et al.  Age-ism: another form of bigotry. , 1969, The Gerontologist.

[71]  R. Kitchin,et al.  Thinking critically about and researching algorithms , 2014, The Social Power of Algorithms.

[72]  Anne Marie Piper,et al.  Going Gray, Failure to Hire, and the Ick Factor: Analyzing How Older Bloggers Talk about Ageism , 2017, CSCW.

[73]  Anne Marie Piper,et al.  A Critical Lens on Dementia and Design in HCI , 2017, CHI.

[74]  Karine Nahon,et al.  Where There is Social Media There is Politics , 2015 .

[75]  L. Christophorou Science , 2018, Emerging Dynamics: Science, Energy, Society and Values.

[76]  Arvind Narayanan,et al.  Semantics derived automatically from language corpora contain human-like biases , 2016, Science.

[77]  Helen Nissenbaum,et al.  Shaping the Web: Why the Politics of Search Engines Matters , 2000, Inf. Soc..

[78]  B. Levy Stereotype Embodiment , 2009, Current directions in psychological science.

[79]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[80]  N. Diakopoulos Algorithmic Accountability Reporting: On the Investigation of Black Boxes , 2014 .

[81]  Joanna N Lahey,et al.  International Comparison of Age Discrimination Laws , 2010, Research on aging.

[82]  Karrie Karahalios,et al.  First I "like" it, then I hide it: Folk Theories of Social Feeds , 2016, CHI.