Generating Fact Checking Explanations

Most existing work on automated fact checking is concerned with predicting the veracity of claims based on metadata, social network spread, language used in claims, and, more recently, evidence supporting or denying claims. A crucial piece of the puzzle that is still missing is to understand how to automate the most elaborate part of the process -- generating justifications for verdicts on claims. This paper provides the first study of how these explanations can be generated automatically based on available claim context, and how this task can be modelled jointly with veracity prediction. Our results indicate that optimising both objectives at the same time, rather than training them separately, improves the performance of a fact checking system. The results of a manual evaluation further suggest that the informativeness, coverage and overall quality of the generated explanations are also improved in the multi-task model.

[1]  Smaranda Muresan,et al.  Robust Document Retrieval and Individual Evidence Modeling for Fact Extraction and Verification. , 2018 .

[2]  Sebastian Riedel,et al.  UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF) , 2018, FEVER@EMNLP.

[3]  Huan Liu,et al.  dEFEND: Explainable Fake News Detection , 2019, KDD.

[4]  Christian Hansen,et al.  MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims , 2019, EMNLP.

[5]  Dominik Stammbach,et al.  Team DOMLIN: Exploiting Evidence Enhancement for the FEVER Shared Task , 2019, EMNLP.

[6]  Richard Socher,et al.  Explain Yourself! Leveraging Language Models for Commonsense Reasoning , 2019, ACL.

[7]  Preslav Nakov,et al.  Automatic Stance Detection Using End-to-End Memory Networks , 2018, NAACL.

[8]  Fan Yang,et al.  XFake: Explainable Fake News Detector with Visualizations , 2019, WWW.

[9]  Thomas Wolf,et al.  DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter , 2019, ArXiv.

[10]  Frank Hutter,et al.  Fixing Weight Decay Regularization in Adam , 2017, ArXiv.

[11]  Andreas Vlachos,et al.  FEVER: a Large-scale Dataset for Fact Extraction and VERification , 2018, NAACL.

[12]  Klaus-Robert Müller,et al.  Evaluating Recurrent Neural Network Explanations , 2019, BlackboxNLP@ACL.

[13]  Hinrich Schütze,et al.  Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement , 2018, ACL.

[14]  Smaranda Muresan,et al.  Where is Your Evidence: Improving Fact-checking by Justification Modeling , 2018 .

[15]  Jiliang Tang,et al.  Multi-Source Multi-Class Fake News Detection , 2018, COLING.

[16]  Kaiming He,et al.  Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour , 2017, ArXiv.

[17]  Chu-Ren Huang,et al.  Fake News Detection Through Multi-Perspective Speaker Profiles , 2017, IJCNLP.

[18]  Christopher Malon,et al.  Team Papelo: Transformer Networks at FEVER , 2019, ArXiv.

[19]  Iryna Gurevych,et al.  UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification , 2018, FEVER@EMNLP.

[20]  Joachim Bingel,et al.  Latent Multi-Task Architecture Learning , 2017, AAAI.

[21]  Bowen Zhou,et al.  SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents , 2016, AAAI.

[22]  Been Kim,et al.  Sanity Checks for Saliency Maps , 2018, NeurIPS.

[23]  Klaus Krippendorff,et al.  Answering the Call for a Standard Reliability Measure for Coding Data , 2007 .

[24]  Frank Hutter,et al.  Decoupled Weight Decay Regularization , 2017, ICLR.

[25]  Mirella Lapata,et al.  Text Summarization with Pretrained Encoders , 2019, EMNLP.

[26]  Akshay Jain,et al.  Fake News Detection , 2018, 2018 IEEE International Students' Conference on Electrical, Electronics and Computer Science (SCEECS).

[27]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[28]  Wei Gao,et al.  Detect Rumor and Stance Jointly by Neural Multi-task Learning , 2018, WWW.

[29]  Martial Hebert,et al.  Cross-Stitch Networks for Multi-task Learning , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Thomas Lukasiewicz,et al.  e-SNLI: Natural Language Inference with Natural Language Explanations , 2018, NeurIPS.

[31]  William Yang Wang.  “Liar, Liar Pants on Fire”: A New Benchmark Dataset for Fake News Detection , 2017, ACL.

[32]  James R. Glass,et al.  Adversarial Domain Adaptation for Stance Detection , 2019, ArXiv.