Societal Biases in Language Generation: Progress and Challenges

Technology for language generation has advanced rapidly, spurred by advancements in pre-training large models on massive amounts of data and the need for intelligent agents to communicate in a natural manner. While techniques can effectively generate fluent text, they can also produce undesirable societal biases that can have a disproportionately negative impact on marginalized populations. Language generation presents unique challenges for biases in terms of direct user interaction and the structure of decoding techniques. To better understand these challenges, we present a survey on societal biases in language generation, focusing on how data and techniques contribute to biases and progress towards reducing biases. Motivated by a lack of studies on biases from decoding techniques, we also conduct experiments to quantify the effects of these techniques. By further discussing general trends and open challenges, we call to attention promising directions for research and the importance of fairness and inclusivity considerations for language generation applications.

[1]  Amandalynne Paullada,et al.  Data and its (dis)contents: A survey of dataset development and use in machine learning research , 2020, Patterns.

[2]  Yejin Choi,et al.  The Curious Case of Neural Text Degeneration , 2019, ICLR.

[3]  Peter Henderson,et al.  Ethical Challenges in Data-Driven Dialogue Systems , 2017, AIES.

[4]  Nam Soo Kim,et al.  Towards Cross-Lingual Generalization of Translation Gender Bias , 2021, FAccT.

[5]  Alex Wang,et al.  BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model , 2019, Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation.

[6]  Marcis Pinnis,et al.  Mitigating Gender Bias in Machine Translation with Target Gender Annotations , 2020, WMT.

[7]  Yann Dauphin,et al.  Hierarchical Neural Story Generation , 2018, ACL.

[8]  Sean A. Munson,et al.  Unequal Representation and Gender Stereotypes in Image Search Results for Occupations , 2015, CHI.

[9]  Nanyun Peng,et al.  The Woman Worked as a Babysitter: On Biases in Language Generation , 2019, EMNLP.

[10]  Nanyun Peng,et al.  Revealing Persona Biases in Dialogue Systems , 2021, ArXiv.

[11]  Richard Socher,et al.  GeDi: Generative Discriminator Guided Sequence Generation , 2021, EMNLP.

[12]  Yoav Goldberg,et al.  Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injection , 2019, Proceedings of the First Workshop on Gender Bias in Natural Language Processing.

[13]  Pascale Fung,et al.  Reducing Gender Bias in Abusive Language Detection , 2018, EMNLP.

[14]  Ben Hutchinson,et al.  Re-imagining Algorithmic Fairness in India and Beyond , 2021, FAccT.

[15]  Noah A. Smith,et al.  Evaluating Gender Bias in Machine Translation , 2019, ACL.

[16]  Yiming Yang,et al.  Transformer-XL: Attentive Language Models beyond a Fixed-Length Context , 2019, ACL.

[17]  Pradyumna Tambwekar,et al.  Towards a Comprehensive Understanding and Accurate Evaluation of Societal Biases in Pre-Trained Transformers , 2021, NAACL.

[18]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[19]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[20]  Jianfeng Gao,et al.  DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation , 2020, ACL.

[21]  Georgiana Dinu,et al.  Improving Gender Translation Accuracy with Filtered Self-Training , 2021, ArXiv.

[22]  Davis Liang,et al.  Decoding and Diversity in Machine Translation , 2020, ArXiv.

[23]  Shikha Bordia,et al.  Identifying and Reducing Gender Bias in Word-Level Language Models , 2019, NAACL.

[24]  Timo Schick,et al.  Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP , 2021, Transactions of the Association for Computational Linguistics.

[25]  Gabriel Stanovsky,et al.  Gender Coreference and Bias Evaluation at WMT 2020 , 2020, WMT@EMNLP.

[26]  Svetlana Kiritchenko,et al.  Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective , 2020, J. Artif. Intell. Res..

[27]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[28]  Mai ElSherief,et al.  Mitigating Gender Bias in Natural Language Processing: Literature Review , 2019, ACL.

[29]  Jiliang Tang,et al.  Does Gender Matter? Towards Fairness in Dialogue Systems , 2020, COLING.

[30]  Anna Farkas,et al.  How to Measure Gender Bias in Machine Translation: Optimal Translators, Multiple Reference Points , 2020, ArXiv.

[31]  Kai-Wei Chang,et al.  “Nice Try, Kiddo”: Investigating Ad Hominems in Dialogue Responses , 2021, NAACL.

[32]  Kellie Webster,et al.  Automatically Identifying Gender Issues in Machine Translation using Perturbations , 2020, EMNLP.

[33]  Yonatan Belinkov,et al.  Investigating Gender Bias in Language Models Using Causal Mediation Analysis , 2020, NeurIPS.

[34]  Po-Sen Huang,et al.  Reducing Sentiment Bias in Language Models via Counterfactual Evaluation , 2019, FINDINGS.

[35]  Colin Raffel,et al.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer , 2019, J. Mach. Learn. Res..

[36]  Li Lucy,et al.  Gender and Representation Bias in GPT-3 Generated Stories , 2021, NUSE.

[37]  Kai-Wei Chang,et al.  BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation , 2021, FAccT.

[38]  Mark O. Riedl,et al.  Reducing Non-Normative Text Generation from Language Models , 2020, INLG.

[39]  Nizar Habash,et al.  Automatic Gender Identification and Reinflection in Arabic , 2019, Proceedings of the First Workshop on Gender Bias in Natural Language Processing.

[40]  Marcus Tomalin,et al.  The practical ethics of bias reduction in machine translation: why domain adaptation is better than data debiasing , 2021, Ethics and Information Technology.

[41]  Catherine Yeo,et al.  Defining and Evaluating Fair Natural Language Generation , 2020, WINLP.

[42]  Zhe Gan,et al.  Distilling Knowledge Learned in BERT for Text Generation , 2019, ACL.

[43]  Miles Brundage,et al.  Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models , 2021, ArXiv.

[44]  Andy Way,et al.  Getting Gender Right in Neural Machine Translation , 2019, EMNLP.

[45]  M. Costa-jussà,et al.  Fine-tuning Neural Machine Translation on Gender-Balanced Datasets , 2020, GEBNLP.

[46]  Bill Byrne,et al.  Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem , 2020, ACL.

[47]  Marco Gaido,et al.  Gender Bias in Machine Translation , 2021, Transactions of the Association for Computational Linguistics.

[48]  William Yang Wang,et al.  They, Them, Theirs: Rewriting with Gender-Neutral English , 2021, ArXiv.

[49]  José A. R. Fonollosa,et al.  Towards Mitigating Gender Bias in a decoder-based Neural Machine Translation model by Adding Contextual Information , 2020, WINLP.

[50]  Jiliang Tang,et al.  Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning , 2020, EMNLP.

[51]  Nathan Srebro,et al.  Equality of Opportunity in Supervised Learning , 2016, NIPS.

[52]  Jason Yosinski,et al.  Plug and Play Language Models: A Simple Approach to Controlled Text Generation , 2020, ICLR.

[53]  Verena Rieser,et al.  Conversational Assistants and Gender Stereotypes: Public Perceptions and Desiderata for Voice Personas , 2020, GEBNLP.

[54]  Houda Bouamor,et al.  Gender-Aware Reinflection using Linguistically Enhanced Neural Models , 2020, GEBNLP.

[55]  Dirk Hovy,et al.  “You Sound Just Like Your Father” Commercial Machine Translation Systems Include Stylistic Biases , 2020, ACL.

[56]  Dirk Hovy,et al.  HONEST: Measuring Hurtful Sentence Completion in Language Models , 2021, NAACL.

[57]  Solon Barocas,et al.  Language (Technology) is Power: A Critical Survey of “Bias” in NLP , 2020, ACL.

[58]  Yusu Qian,et al.  Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function , 2019, ACL.

[59]  Jason Weston,et al.  Multi-Dimensional Gender Bias Classification , 2020, EMNLP.

[60]  William Yang Wang,et al.  The Truth is Out There: Investigating Conspiracy Theories in Text Generation , 2021, ArXiv.

[61]  Elias Benussi,et al.  How True is GPT-2? An Empirical Analysis of Intersectional Occupational Biases , 2021, ArXiv.

[62]  Sameer Singh,et al.  Universal Adversarial Triggers for Attacking and Analyzing NLP , 2019, EMNLP.

[63]  Diyi Yang,et al.  Automatically Neutralizing Subjective Bias in Text , 2019, AAAI.

[64]  Emily M. Bender,et al.  On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 , 2021, FAccT.

[65]  Luís C. Lamb,et al.  Assessing gender bias in machine translation: a case study with Google Translate , 2018, Neural Computing and Applications.

[66]  Nanyun Peng,et al.  Towards Controllable Biases in Language Generation , 2020, FINDINGS.

[67]  Eric Gilbert,et al.  VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text , 2014, ICWSM.

[68]  Steven Bird,et al.  Decolonising Speech and Language Technology , 2020, COLING.

[69]  Jason Weston,et al.  Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation , 2020, EMNLP.

[70]  Nam Soo Kim,et al.  On Measuring Gender Bias in Translation of Gender-neutral Pronouns , 2019, Proceedings of the First Workshop on Gender Bias in Natural Language Processing.

[71]  Rachel Rudinger,et al.  “You Are Grounded!”: Latent Name Artifacts in Pre-trained Language Models , 2020, EMNLP.

[72]  B. Byrne,et al.  Neural Machine Translation Doesn’t Translate Gender Coreference Right Unless You Make It , 2020, GEBNLP.

[73]  Yejin Choi,et al.  PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction , 2020, EMNLP.

[74]  Colin Raffel,et al.  Extracting Training Data from Large Language Models , 2020, USENIX Security Symposium.

[75]  Ryan Cotterell,et al.  Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology , 2019, ACL.

[76]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[77]  Yejin Choi,et al.  On-the-Fly Controlled Text Generation with Experts and Anti-Experts , 2021, ArXiv.

[78]  Bill Byrne,et al.  First the worst: Finding better gender translations during beam search , 2021, ArXiv.

[79]  Timnit Gebru,et al.  Datasheets for datasets , 2018, Commun. ACM.

[80]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[81]  Marta R. Costa-jussà,et al.  Equalizing Gender Bias in Neural Machine Translation with Word Embeddings Techniques , 2019, Proceedings of the First Workshop on Gender Bias in Natural Language Processing.

[82]  Emily Sheng,et al.  Investigating Societal Biases in a Poetry Composition System , 2020, GEBNLP.

[83]  Anupam Datta,et al.  Gender Bias in Neural Natural Language Processing , 2018, Logic, Language, and Security.

[84]  Ahmed Y. Tawfik,et al.  Gender aware spoken language translation applied to English-Arabic , 2018, 2018 2nd International Conference on Natural Language and Speech Processing (ICNLSP).

[85]  Emily M. Bender,et al.  Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science , 2018, TACL.

[86]  James Zou,et al.  Persistent Anti-Muslim Bias in Large Language Models , 2021, AIES.

[87]  Adithya Renduchintala,et al.  Investigating Failures of Automatic Translation in the Case of Unambiguous Gender , 2021, ACL.

[88]  Alec Radford,et al.  Improving Language Understanding by Generative Pre-Training , 2018 .

[89]  Natalia Criado,et al.  Discovering and Categorising Language Biases in Reddit , 2020, ICWSM.

[90]  Percy Liang,et al.  Fairness Without Demographics in Repeated Loss Minimization , 2018, ICML.

[91]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[92]  Alec Radford,et al.  Release Strategies and the Social Impacts of Language Models , 2019, ArXiv.

[93]  Dirk Hovy,et al.  Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview , 2019, ACL.

[94]  L. Elisa Celis,et al.  Dialect Diversity in Text Summarization on Twitter , 2020, WWW.

[95]  Carlos Escolano,et al.  Gender Bias in Multilingual Neural Machine Translation: The Architecture Matters , 2020, ArXiv.

[96]  Zeyu Li,et al.  Learning Gender-Neutral Word Embeddings , 2018, EMNLP.

[97]  Saif Mohammad,et al.  Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems , 2018, *SEMEVAL.