The Psychosocial Impacts of Generative AI Harms

The rapid emergence of generative Language Models (LMs) has led to growing concern about the impacts that their unexamined adoption may have on the social well-being of diverse user groups. Meanwhile, LMs are increasingly being adopted in K-20 schools and one-on-one student settings with minimal investigation of potential harms associated with their deployment. Motivated in part by real-world/everyday use cases (e.g., an AI writing assistant) this paper explores the potential psychosocial harms of stories generated by five leading LMs in response to open-ended prompting. We extend findings of stereotyping harms analyzing a total of 150K 100-word stories related to student classroom interactions. Examining patterns in LM-generated character demographics and representational harms (i.e., erasure, subordination, and stereotyping) we highlight particularly egregious vignettes, illustrating the ways LM-generated outputs may influence the experiences of users with marginalized and minoritized identities, and emphasizing the need for a critical understanding of the psychosocial impacts of generative AI tools when deployed and utilized in diverse social contexts.

[1]  Catherine Riegle-Crumb,et al.  Sex and gender essentialism in textbooks , 2024, Science.

[2]  Andy Extance.  ChatGPT has entered the classroom: how LLMs could transform education , 2023, Nature.

[3]  R. Behringer,et al.  Asian Americans in STEM are not a monolith , 2023, Cell.

[4]  Sarah L. Eddy,et al.  “It’s completely erasure”: A Qualitative Exploration of Experiences of Transgender, Nonbinary, Gender Nonconforming, and Questioning Students in Biology Courses , 2022, CBE life sciences education.

[5]  Negar Rostamzadeh,et al.  Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction , 2022, AIES.

[6]  Nazanin Andalibi,et al.  Algorithmic Folk Theories and Identity: How TikTok Users Co-Produce Knowledge of Identity and Engage in Algorithmic Resistance , 2021, Proc. ACM Hum. Comput. Interact..

[7]  Arianne E. Eason,et al.  Representations of Native Americans in U.S. culture? A case of omissions and commissions , 2021, The Social Science Journal.

[8]  Kai-Wei Chang,et al.  On Measures of Biases and Harms in NLP , 2021, AACL/IJCNLP.

[9]  Cassidy R. Sugimoto,et al.  Avoiding bias when inferring race using name-based approaches , 2021, PloS one.

[10]  Emily Denton,et al.  Towards a critical race methodology in algorithmic fairness , 2019, FAT*.

[11]  Ebony O. McGee,et al.  “Black Genius, Asian Fail”: The Detriment of Stereotype Lift and Stereotype Threat in High-Achieving Asian and Black STEM Students , 2018, AERA Open.

[12]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[13]  Arianne E. Eason,et al.  Making the Invisible Visible: Acts of Commission and Omission , 2017 .

[14]  Charissa S. L. Cheah,et al.  Moving beyond the model minority , 2017 .

[15]  G. Walton,et al.  Stereotype Threat in Organizations: Implications for Equity and Performance , 2015 .

[16]  J. Cammarota.  Blindsided by the Avatar: White Saviors and Allies Out of Hollywood and in Education , 2011 .

[17]  Stephanie A. Fryberg,et al.  Of Warrior Chiefs and Indian Princesses: The Psychological Consequences of American Indian Mascots , 2008 .

[18]  M. Tappan.  Refraining Internalized Oppression and Internalized Domination: From the Psychological to the Sociocultural , 2006, Teachers College Record: The Voice of Scholarship in Education.

[19]  Nalini Ambady,et al.  “Math is Hard!” The effect of gender priming on women’s attitudes , 2006 .

[20]  John M. Reveles,et al.  Scientific literacy and discursive identity: A theoretical framework for understanding science learning , 2005 .

[21]  N. Ambady,et al.  Stereotype Susceptibility: Identity Salience and Shifts in Quantitative Performance , 1999 .

[22]  C. Steele.  A threat in the air. How stereotypes shape intellectual identity and performance. , 1997, The American psychologist.

[23]  C. Steele,et al.  Stereotype threat and the intellectual test performance of African Americans. , 1995, Journal of personality and social psychology.

[24]  K. Crenshaw.  Mapping the margins: intersectionality, identity politics, and violence against women of color , 1991 .

[25]  Stephanie A. Fryberg,et al.  “A Future Denied” for Young Indigenous People: From Social Disruption to Possible Futures , 2019, Handbook of Indigenous Education.

[26]  S. Spencer,et al.  Stereotype Threat. , 2016, Annual review of psychology.