No NLP Task Should be an Island: Multi-disciplinarity for Diversity in News Recommender Systems

Natural Language Processing (NLP) is defined by specific, separate tasks, with each their own literature, benchmark datasets, and definitions. In this position paper, we argue that for a complex problem such as the threat to democracy by non-diverse news recommender systems, it is important to take into account a higher-order, normative goal and its implications. Experts in ethics, political science and media studies have suggested that news recommendation systems could be used to support a deliberative democracy. We reflect on the role of NLP in recommendation systems with this specific goal in mind and show that this theory of democracy helps to identify which NLP tasks and techniques can support this goal, and what work still needs to be done. This leads to recommendations for NLP researchers working on this specific problem as well as researchers working on other complex multidisciplinary problems.

[1]  J. Cappella,et al.  Echo Chamber: Rush Limbaugh and the Conservative Media Establishment , 2008 .

[2]  Natali Helberger,et al.  Challenged by news personalisation: five perspectives on the right to receive information , 2017 .

[3]  Mark Carlebach,et al.  News Aggregation with Diverse Viewpoint Identification Using Neural Embeddings and Semantic Understanding Models , 2020, ARGMINING.

[4]  Engin Bozdag,et al.  Bias in algorithmic filtering and personalization , 2013, Ethics and Information Technology.

[5]  N. Helberger.  On the Democratic Role of News Recommenders , 2019, Algorithms, Automation, and News.

[6]  Yi-Cheng Zhang,et al.  Solving the apparent diversity-accuracy dilemma of recommender systems , 2008, Proceedings of the National Academy of Sciences.

[7]  Sameer Singh,et al.  Beyond Accuracy: Behavioral Testing of NLP Models with CheckList , 2020, ACL.

[8]  Sandra Wachter.  Affinity Profiling and Discrimination by Association in Online Behavioural Advertising , 2019, SSRN Electronic Journal.

[9]  Saif Mohammad,et al.  Stance and Sentiment in Tweets , 2016, ACM Trans. Internet Techn..

[10]  Iryna Gurevych,et al.  Parsing Argumentation Structures in Persuasive Essays , 2016, CL.

[11]  Alvin Zhou.  #Republic: Divided Democracy in the Age of Social Media , 2017 .

[12]  Dan Goldwasser,et al.  Weakly Supervised Learning of Nuanced Frames for Analyzing Polarization in News Media , 2020, EMNLP.

[13]  Antske Fokkens,et al.  BiographyNet: Methodological Issues when NLP supports historical research , 2014, LREC.

[14]  Benno Stein,et al.  Computational Argumentation Quality Assessment in Natural Language , 2017, EACL.

[15]  Robb Willer,et al.  The Social Structure of Political Echo Chambers: Variation in Ideological Homophily in Online Networks , 2017 .

[16]  Chantal van Son,et al.  Annotating Perspectives on Vaccination , 2020, LREC.

[17]  Natali Helberger,et al.  Recommenders with a Mission: Assessing Diversity in News Recommendations , 2020, CHIIR.

[18]  Tim Draws,et al.  Helping users discover perspectives: Enhancing opinion mining with joint topic models , 2020, 2020 International Conference on Data Mining Workshops (ICDMW).

[19]  Eli Pariser,et al.  The Filter Bubble: What the Internet Is Hiding from You , 2011 .

[20]  Iryna Gurevych,et al.  A Retrospective Analysis of the Fake News Challenge Stance-Detection Task , 2018, COLING.

[21]  Bernard Manin,et al.  On Legitimacy and Political Deliberation , 1987 .

[22]  Chantal van Son,et al.  GRaSP: A Multilayered Annotation Scheme for Perspectives , 2016, LREC.

[23]  Damian Trilling,et al.  Should We Worry About Filter Bubbles? , 2016 .

[24]  Nigel Collier,et al.  STANDER: An Expert-Annotated Dataset for News Stance Detection and Evidence Retrieval , 2020, FINDINGS.

[25]  Zizi Papacharissi,et al.  The virtual sphere: The internet as a public sphere , 2018, New Media Soc..

[26]  Els Lefever,et al.  Annotating Topics, Stance, Argumentativeness and Claims in Dutch Social Media Comments: A Pilot Study , 2020, ARGMINING.

[27]  Nava Tintarev,et al.  Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics , 2021, SIGKDD Explor..

[28]  Damian Trilling,et al.  Do not blame it on the algorithm: an empirical assessment of multiple recommender systems and their impact on content diversity , 2018 .

[29]  Kathleen McKeown,et al.  Zero-Shot Stance Detection: A Dataset and Model Using Generalized Topic Representations , 2020, EMNLP.

[30]  Suzan Verberne,et al.  The reach of commercially motivated junk news on Facebook , 2019, PloS one.

[31]  Sameer Singh,et al.  COVIDLies: Detecting COVID-19 Misinformation on Social Media , 2020, NLP4COVID@EMNLP.

[32]  Dietmar Jannach,et al.  News recommender systems - Survey and roads ahead , 2018, Inf. Process. Manag..

[33]  Emily M. Bender,et al.  On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 , 2021, FAccT.

[34]  Antske Fokkens,et al.  Studying Muslim Stereotyping through Microportrait Extraction , 2018, LREC.

[35]  Derek Bridge,et al.  Diversity, Serendipity, Novelty, and Coverage , 2016, ACM Trans. Interact. Intell. Syst..

[36]  Wolfgang Schulz,et al.  Diversity, Fairness, and Data-Driven Personalization in (News) Recommender System (Dagstuhl Perspectives Workshop 19482) , 2019, Dagstuhl Reports.

[37]  Preslav Nakov,et al.  Findings of the NLP4IF-2019 Shared Task on Fine-Grained Propaganda Detection , 2019, EMNLP.

[38]  Nava Tintarev,et al.  Presenting Diversity Aware Recommendations: Making Challenging News Acceptable , 2017 .

[39]  Feng Lu,et al.  Beyond Optimizing for Clicks: Incorporating Editorial Values in News Recommendation , 2020, UMAP.