Political Depolarization of News Articles Using Attribute-aware Word Embeddings

Political polarization in the US is on the rise. This polarization negatively affects the public sphere by contributing to the creation of ideological echo chambers. In this paper, we focus on addressing one of the factors that contributes to this polarity, polarized media. We introduce a framework for depolarizing news articles. Given an article on a certain topic with a particular ideological slant (eg., liberal or conservative), the framework first detects polar language in the article and then generates a new article with the polar language replaced with neutral expressions. To detect polar words, we train a multiattribute-aware word embedding model that is aware of ideology and topics on 360k full-length media articles. Then, for text generation, we propose a new algorithm called Text Annealing Depolarization Algorithm (TADA). TADA retrieves neutral expressions from the word embedding model that not only decrease ideological polarity but also preserve the original argument of the text, while maintaining grammatical correctness. We evaluate our framework by comparing the depolarized output of our model in two modes, fully-automatic and semi-automatic, on 99 stories spanning 11 topics. Based on feedback from 161 human testers, our framework successfully depolarized 90.1% of paragraphs in semi-automatic mode and 78.3% of paragraphs in fully-automatic mode. Furthermore, 81.2% of the testers agree that the non-polar content information is well-preserved and 79% agree that depolarization does not harm semantic correctness when they compare the original text and the depolarized text. Our work shows that data-driven methods can help to locate political polarity and aid in the depolarization of articles.

[1]  Ayush Singh,et al.  Sentiment Transfer using Seq2Seq Adversarial Autoencoders , 2018, ArXiv.

[2]  Xuanjing Huang,et al.  Adversarial Multi-Criteria Learning for Chinese Word Segmentation , 2017, ACL.

[3]  Yi Yang,et al.  Overcoming Language Variation in Sentiment Analysis with Social Attention , 2015, TACL.

[4]  Benno Stein,et al.  Learning to Flip the Bias of News Headlines , 2018, INLG.

[5]  Lili Mou,et al.  Disentangled Representation Learning for Non-Parallel Text Style Transfer , 2018, ACL.

[6]  Guillaume Lample,et al.  Multiple-Attribute Text Style Transfer , 2018, ArXiv.

[7]  Philip Resnik,et al.  Political Ideology Detection Using Recursive Neural Networks , 2014, ACL.

[8]  Sharon Goldwater,et al.  Aye or naw, whit dae ye hink? Scottish independence and linguistic identity on social media , 2017, EACL.

[9]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[10]  David Lazer,et al.  More Voices Than Ever? Quantifying Media Bias in Networks , 2011, ICWSM.

[11]  Guillaume Lample,et al.  Multiple-Attribute Text Rewriting , 2018, ICLR.

[12]  Amber E. Boydstun,et al.  Media Framing of Capital Punishment and Its Impact on Individuals' Cognitive Responses , 2008 .

[13]  Graham Neubig,et al.  Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction , 2017, ACL.

[14]  Furu Wei,et al.  Dictionary-Guided Editing Networks for Paraphrase Generation , 2018, AAAI.

[15]  Zhou Yu,et al.  Structured Content Preservation for Unsupervised Text Style Transfer , 2018, ArXiv.

[16]  Thorsten Joachims,et al.  Evaluation methods for unsupervised word embeddings , 2015, EMNLP.

[17]  A. Tversky,et al.  Choices, Values, and Frames , 2000 .

[18]  Yulia Tsvetkov,et al.  Style Transfer Through Back-Translation , 2018, ACL.

[19]  Kevin Gimpel,et al.  ALBERT: A Lite BERT for Self-supervised Learning of Language Representations , 2019, ICLR.

[20]  Chen Wu,et al.  A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer , 2019, ACL.

[21]  Élise Huillery.  History Matters: The Long-Term Impact of Colonial Public Investments in French West Africa , 2009 .

[22]  Roger Levy,et al.  Simple dynamic word embeddings for mapping perceptions in the public sphere , 2019, Proceedings of the Third Workshop on Natural Language Processing and Computational Social Science.

[23]  Percy Liang,et al.  Delete, Retrieve, Generate: a Simple Approach to Sentiment and Style Transfer , 2018, NAACL.

[24]  Regina Barzilay,et al.  Style Transfer from Non-Parallel Text by Cross-Alignment , 2017, NIPS.

[25]  Mani B. Srivastava,et al.  Generating Natural Language Adversarial Examples , 2018, EMNLP.

[26]  Xuanjing Huang,et al.  Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation , 2019, ACL.

[27]  Percy Liang,et al.  Generating Sentences by Editing Prototypes , 2017, TACL.

[28]  Samy Bengio,et al.  Content preserving text generation with attribute controls , 2018, NeurIPS.

[29]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[30]  Noah A. Smith,et al.  Shedding (a Thousand Points of) Light on Biased Language , 2010, Mturk@HLT-NAACL.

[31]  Tim Groseclose,et al.  A Measure of Media Bias , 2005 .

[32]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[33]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[34]  Wanxiang Che,et al.  Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency , 2019, ACL.

[35]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[36]  Tommi S. Jaakkola,et al.  Sequence to Better Sequence: Continuous Revision of Combinatorial Structures , 2017, ICML.

[37]  Jinjun Xiong,et al.  Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus , 2019, NAACL.

[39]  John Marangos,et al.  The Evolution of the Anti-Washington Consensus Debate: From ‘Post-Washington Consensus’ to ‘After the Washington Consensus’ , 2008 .

[40]  Zhe Gan,et al.  Adversarial Text Generation via Feature-Mover's Distance , 2018, NeurIPS.