Automated Fact-Checking for Assisting Human Fact-Checkers

The reporting and analysis of current events around the globe has expanded from professional, editorlead journalism all the way to citizen journalism. Politicians and other key players enjoy direct access to their audiences through social media, bypassing the filters of official cables or traditional media. However, the multiple advantages of free speech and direct communication are dimmed by the misuse of the media to spread inaccurate or misleading claims. These phenomena have led to the modern incarnation of the fact-checker — a professional whose main aim is to examine claims using available evidence to assess their veracity. As in other text forensics tasks, the amount of information available makes the work of the fact-checker more difficult. With this in mind, starting from the perspective of the professional fact-checker, we survey the available intelligent technologies that can support the human expert in the different steps of her fact-checking endeavor. These include identifying claims worth fact-checking; detecting relevant previously fact-checked claims; retrieving relevant evidence to fact-check a claim; and actually verifying a claim. In each case, we pay attention to the challenges in future work and the potential impact on real-world fact-checking.

[1]  Emily B. Fox,et al.  A Bayesian Approach for Predicting the Popularity of Tweets , 2013, ArXiv.

[2]  Smaranda Muresan,et al.  Where is Your Evidence: Improving Fact-checking by Justification Modeling , 2018 .

[3]  Preslav Nakov,et al.  A Survey on Computational Propaganda Detection , 2020, IJCAI.

[4]  Zita Marinho,et al.  Automated Fact Checking in the News Room , 2019, WWW.

[5]  Bo Zhao,et al.  A Survey on Truth Discovery , 2015, SKDD.

[6]  Preslav Nakov,et al.  Fact-Checking Meets Fauxtography: Verifying Claims About Images , 2019, EMNLP.

[7]  Eunsol Choi,et al.  Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking , 2017, EMNLP.

[8]  Jisun An,et al.  A Survey on Predicting the Factuality and the Bias of News Media , 2021, ArXiv.

[9]  Firoj Alam,et al.  The Role of Context in Detecting Previously Fact-Checked Claims , 2021, ArXiv.

[10]  Iryna Gurevych,et al.  UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification , 2018, FEVER@EMNLP.

[11]  Wenhu Chen,et al.  TabFact: A Large-scale Dataset for Table-based Fact Verification , 2019, ICLR.

[12]  L. Christophorou Science , 2018, Emerging Dynamics: Science, Energy, Society and Values.

[13]  Andreas Vlachos,et al.  The Fact Extraction and VERification (FEVER) Shared Task , 2018, ArXiv.

[14]  Firoj Alam,et al.  A Survey on Multimodal Disinformation Detection , 2021, ArXiv.

[15]  Alberto Barrón-Cedeño,et al.  The CLEF-2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News , 2021, ECIR.

[16]  Andrew G. Glen,et al.  APPL , 2001 .

[17]  Haonan Chen,et al.  Combining Fact Extraction and Verification with Neural Semantic Matching Networks , 2018, AAAI.

[18]  Isabelle Augenstein,et al.  Fact Check-Worthiness Detection as Positive Unlabelled Learning , 2020, EMNLP.

[19]  Jimmy J. Lin,et al.  Cross-Domain Modeling of Sentence-Level Evidence for Document Retrieval , 2019, EMNLP.

[20]  Preslav Nakov,et al.  It Takes Nine to Smell a Rat: Neural Multi-Task Learning for Check-Worthiness Prediction , 2019, RANLP.

[21]  Maram Hasanain,et al.  bigIR at CheckThat! 2020: Multilingual BERT for Ranking Arabic Tweets by Check-worthiness , 2020, CLEF.

[22]  Preslav Nakov,et al.  Team Alex at CLEF CheckThat! 2020: Identifying Check-Worthy Tweets With Transformer Models , 2020, CLEF.

[23]  Preslav Nakov,et al.  A Context-Aware Approach for Detecting Worth-Checking Claims in Political Debates , 2017, RANLP.

[24]  Ben Adler,et al.  Real-time Claim Detection from News Articles and Retrieval of Semantically-Similar Factchecks , 2019, NewsIR@SIGIR.

[25]  Chengkai Li,et al.  Toward Automated Fact-Checking: Detecting Check-worthy Factual Claims by ClaimBuster , 2017, KDD.

[26]  Henning Wachsmuth,et al.  Extractive Snippet Generation for Arguments , 2020, SIGIR.

[27]  Gerhard Weikum,et al.  Credibility Assessment of Textual Claims on the Web , 2016, CIKM.

[28]  Gerhard Weikum,et al.  Tracy: Tracing Facts over Knowledge Graphs and Text , 2019, WWW.

[29]  Neema Kotonya,et al.  Explainable Automated Fact-Checking: A Survey , 2020, COLING.

[30]  Paolo Papotti,et al.  Scrutinizer , 2020, Proc. VLDB Endow..

[31]  Kyumin Lee,et al.  The Rise of Guardians: Fact-checking URL Recommendation to Combat Fake News , 2018, SIGIR.

[32]  J. Meigs,et al.  WHO Technical Report , 1954, The Yale Journal of Biology and Medicine.

[33]  Jacob Daniel Devasier,et al.  Gradient-Based Adversarial Training on Transformer Networks for Detecting Check-Worthy Factual Claims , 2020, ArXiv.

[34]  Paolo Papotti,et al.  Explainable Fact Checking with Probabilistic Answer Set Programming , 2019, TTO.

[35]  Chengkai Li,et al.  Detecting Check-worthy Factual Claims in Presidential Debates , 2015, CIKM.

[36]  Editors , 1986, Brain Research Bulletin.

[37]  Barbara Poblete,et al.  Information credibility on twitter , 2011, WWW.

[38]  Preslav Nakov,et al.  Overview of CheckThat 2020: Automatic Identification and Verification of Claims in Social Media , 2020, CLEF.

[39]  Damian Jimenez,et al.  ClaimPortal: Integrated Monitoring, Searching, Checking, and Analytics of Factual Claims on Twitter , 2019, ACL.

[40]  Preslav Nakov,et al.  Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthiness , 2018, CLEF.

[41]  Rafael Vieira,et al.  Can Machines Learn to Detect Fake News? A Survey Focused on Social Media , 2019, HICSS.

[42]  Preslav Nakov,et al.  ClaimRank: Detecting Check-Worthy Claims in Arabic and English , 2018, NAACL.

[43]  Mucahid Kutlu,et al.  Too Many Claims to Fact-Check: Prioritizing Political Claims Based on Check-Worthiness , 2020, CIKM.

[44]  Preslav Nakov,et al.  CheckThat! at CLEF 2019: Automatic Identification and Verification of Claims , 2019, ECIR.

[45]  Kyumin Lee,et al.  Where Are the Facts? Searching for Fact-checked Information to Alleviate the Spread of Fake News , 2020, EMNLP.

[46]  Andreas Vlachos,et al.  Automated Fact Checking: Task Formulations, Methods and Future Directions , 2018, COLING.

[47]  Fabio Petroni,et al.  Generating Fact Checking Briefs , 2020, EMNLP.

[48]  Preslav Nakov,et al.  That is a Known Lie: Detecting Previously Fact-Checked Claims , 2020, ACL.

[49]  Arkaitz Zubiaga,et al.  Towards Automated Factchecking: Developing an Annotation Schema and Benchmark for Consistent Automated Claim Detection , 2018, ArXiv.

[50]  Emily M. Bender,et al.  On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 , 2021, FAccT.

[51]  Madian Khabsa,et al.  Towards Few-shot Fact-Checking via Perplexity , 2021, NAACL.

[52]  Preslav Nakov,et al.  CheckThat! at CLEF 2020: Enabling the Automatic Identification and Verification of Claims in Social Media , 2020, ECIR.

[53]  Vinay Setty,et al.  BRENDA: Browser Extension for Fake News Detection , 2020, SIGIR.

[54]  Paul Rodrigues,et al.  Accenture at CheckThat! 2020: If you say so: Post-hoc fact-checking of Claims using Transformer-based Models , 2020, CLEF.

[55]  Andreas Vlachos,et al.  Fact Checking: Task definition and dataset construction , 2014, LTCSS@ACL.

[56]  Preslav Nakov,et al.  FANG: Leveraging Social Context for Fake News Detection Using Graph Representation , 2020, CIKM.

[57]  LiakataMaria,et al.  Detection and Resolution of Rumours in Social Media , 2018 .

[58]  Christian Hansen,et al.  MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims , 2019, EMNLP.

[59]  Christopher Malon,et al.  Team Papelo: Transformer Networks at FEVER , 2019, ArXiv.

[60]  Xianzhi Wang,et al.  Deep learning for misinformation detection on online social networks: a survey and new perspectives , 2020, Social Network Analysis and Mining.

[61]  Sinan Aral,et al.  The spread of true and false news online , 2018, Science.

[62]  Suhang Wang,et al.  Fake News Detection on Social Media: A Data Mining Perspective , 2017, SKDD.