E-BART: Jointly Predicting and Explaining Truthfulness

Automated fact-checking (AFC) systems exist to combat disinformation, however their complexity makes them opaque to the end user, making it difficult to foster trust. In this paper, we introduce the E-BART model with the hope of making progress on this front. E-BART is able to provide a veracity prediction for a claim, and jointly generate a human-readable explanation for this decision. We show that E-BART is competitive with the state-of-theart on the e-FEVER and e-SNLI tasks. In addition, we validate the joint-prediction architecture by showing 1) that generating explanations does not significantly impede the model from performing well in its main task of veracity prediction, and 2) that predicted veracity and explanations are more internally coherent when generated jointly than separately. Finally, we also conduct human evaluations on the impact of generated explanations and observe that explanations increase human ability to spot misinformation and make people more skeptical about claims.

[1]  Omer Levy,et al.  BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension , 2019, ACL.

[2]  Jin-Hyuk Hong,et al.  Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information , 2018, AAAI.

[3]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[4]  Huan Liu,et al.  dEFEND: Explainable Fake News Detection , 2019, KDD.

[5]  Andreas Vlachos,et al.  The Fact Extraction and VERification (FEVER) Shared Task , 2018, FEVER@EMNLP.

[6]  Elliott Ash,et al.  e-FEVER: Explanations and Summaries forAutomated Fact Checking , 2020, TTO.

[7]  Christopher Potts,et al.  A large annotated corpus for learning natural language inference , 2015, EMNLP.

[8]  Jonathan Pilault,et al.  Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data , 2020, ArXiv.

[9]  Gianluca Demartini,et al.  Can The Crowd Identify Misinformation Objectively?: The Effects of Judgment Scale and Assessor's Background , 2020, SIGIR.

[10]  Ambreen Nazir,et al.  DTCA: Decision Tree-based Co-Attention Networks for Explainable Claim Verification , 2020, ACL.

[11]  David G. Rand,et al.  Shifting attention to accuracy can reduce misinformation online , 2021, Nature.

[12]  Hai Zhao,et al.  Semantics-aware BERT for Language Understanding , 2020, AAAI.

[13]  Thomas Wolf,et al.  DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter , 2019, ArXiv.

[14]  Alec Radford,et al.  Improving Language Understanding by Generative Pre-Training , 2018 .

[15]  Giuseppe Serra,et al.  Distilling the Evidence to Augment Fact Verification Models , 2020, FEVER.

[16]  Lucas Graves.  Understanding the Promise and Limits of Automated Fact-Checking , 2018 .

[17]  Dominik Stammbach,et al.  Team DOMLIN: Exploiting Evidence Enhancement for the FEVER Shared Task , 2019, EMNLP.

[18]  Haonan Chen,et al.  Combining Fact Extraction and Verification with Neural Semantic Matching Networks , 2018, AAAI.

[19]  Xiaodong Liu,et al.  Multi-Task Deep Neural Networks for Natural Language Understanding , 2019, ACL.

[20]  Paolo Papotti,et al.  Explainable Fact Checking with Probabilistic Answer Set Programming , 2019, TTO.

[21]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[22]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[23]  Iryna Gurevych,et al.  UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification , 2018, FEVER@EMNLP.

[24]  Ronald Denaux,et al.  Linked Credibility Reviews for Explainable Misinformation Detection , 2020, SEMWEB.

[25]  Mucahid Kutlu,et al.  Annotator Rationales for Labeling Tasks in Crowdsourcing , 2020, J. Artif. Intell. Res..

[26]  Marcel Worring,et al.  BERT for Evidence Retrieval and Claim Verification , 2019, ECIR.

[27]  Andreas Vlachos,et al.  Fact Checking: Task definition and dataset construction , 2014, LTCSS@ACL.

[28]  Sebastian Riedel,et al.  UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF) , 2018, FEVER@EMNLP.

[29]  Thomas Lukasiewicz,et al.  e-SNLI: Natural Language Inference with Natural Language Explanations , 2018, NeurIPS.

[30]  Ehsan Toreini,et al.  The relationship between trust in AI and trustworthy machine learning technologies , 2019, FAT*.

[31]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[32]  Francesca Toni,et al.  Explainable Automated Fact-Checking for Public Health Claims , 2020, EMNLP.

[33]  Jakob Grue Simonsen,et al.  Generating Fact Checking Explanations , 2020, ACL.