Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing

Explainable Natural Language Processing (EXNLP) has increasingly focused on 1 collecting human-annotated textual explanations. These explanations are used 2 downstream in three ways: as data augmentation to improve performance on a 3 predictive task, as supervision to train models to produce explanations for their 4 predictions, and as a ground-truth to evaluate model-generated explanations. In 5 this review, we identify 61 datasets with three predominant classes of textual expla6 nations (highlights, free-text, and structured), organize the literature on annotating 7 each type, identify strengths and shortcomings of existing collection methodologies, 8 and give recommendations for collecting EXNLP datasets in the future. 9

[1]  Yejin Choi,et al.  Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs , 2020, FINDINGS.

[2]  Barry Smyth,et al.  Generating Plausible Counterfactual Explanations for Deep Transformers in Financial Text Classification , 2020, COLING.

[3]  Eunsol Choi,et al.  QED: A Framework and Dataset for Explanations in Question Answering , 2020, Transactions of the Association for Computational Linguistics.

[4]  Marco Valentino,et al.  A Survey on Explainability in Machine Reading Comprehension , 2020, ArXiv.

[5]  Tommi S. Jaakkola,et al.  Towards Robust Interpretability with Self-Explaining Neural Networks , 2018, NeurIPS.

[6]  Ye Zhang,et al.  Rationale-Augmented Convolutional Neural Networks for Text Classification , 2016, EMNLP.

[7]  Ellie Pavlick,et al.  Inherent Disagreements in Human Textual Inferences , 2019, Transactions of the Association for Computational Linguistics.

[8]  Doug Downey,et al.  Abductive Commonsense Reasoning , 2019, ICLR.

[9]  Sawan Kumar,et al.  NILE : Natural Language Inference with Faithful Natural Language Explanations , 2020, ACL.

[10]  Ido Dagan,et al.  Controlled Crowdsourcing for High-Quality QA-SRL Annotation , 2019, ACL.

[11]  Yoav Goldberg,et al.  Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness? , 2020, ACL.

[12]  H. Hastie,et al.  A Survey of Explainable AI Terminology , 2019, Proceedings of the 1st Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence (NL4XAI 2019).

[13]  Chandan Singh,et al.  Definitions, methods, and applications in interpretable machine learning , 2019, Proceedings of the National Academy of Sciences.

[14]  Iryna Gurevych,et al.  A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking , 2019, CoNLL.

[15]  Xiang Zhou,et al.  What Can We Learn from Collective Human Opinions on Natural Language Inference Data? , 2020, EMNLP.

[16]  Qiaozhu Mei,et al.  Extractive Adversarial Networks: High-Recall Explanations for Identifying Personal Attacks in Social Media Posts , 2018, EMNLP.

[17]  Francesca Toni,et al.  Explainable Automated Fact-Checking for Public Health Claims , 2020, EMNLP.

[18]  Amandalynne Paullada,et al.  Data and its (dis)contents: A survey of dataset development and use in machine learning research , 2020, Patterns.

[19]  Diyi Yang,et al.  ToTTo: A Controlled Table-To-Text Generation Dataset , 2020, EMNLP.

[20]  Georg Groh,et al.  Investigating Annotator Bias with a Graph-Based Approach , 2020, ALW.

[21]  Dan Roth,et al.  Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences , 2018, NAACL.

[22]  Yejin Choi,et al.  Social Bias Frames: Reasoning about Social and Power Implications of Language , 2020, ACL.

[23]  Jason Weston,et al.  ELI5: Long Form Question Answering , 2019, ACL.

[24]  Animesh Mukherjee,et al.  HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection , 2020, AAAI.

[25]  Smaranda Muresan,et al.  Where is Your Evidence: Improving Fact-checking by Justification Modeling , 2018 .

[26]  Tom M. Mitchell,et al.  Joint Concept Learning and Semantic Parsing from Natural Language Explanations , 2017, EMNLP.

[27]  Iain J. Marshall,et al.  Evidence Inference 2.0: More Data, Better Models , 2020, BIONLP.

[28]  Fan Yang,et al.  Evaluating Explanation Without Ground Truth in Interpretable Machine Learning , 2019, ArXiv.

[29]  Jian Zhang,et al.  SQuAD: 100,000+ Questions for Machine Comprehension of Text , 2016, EMNLP.

[30]  Chenhao Tan,et al.  What Gets Echoed? Understanding the “Pointers” in Explanations of Persuasive Arguments , 2019, EMNLP.

[31]  Reut Tsarfaty,et al.  The Extraordinary Failure of Complement Coercion Crowdsourcing , 2020, INSIGHTS.

[32]  Lora Aroyo,et al.  Truth Is a Lie: Crowd Truth and the Seven Myths of Human Annotation , 2015, AI Mag..

[33]  Dinesh Garg,et al.  Explanations for CommonsenseQA: New Dataset and Models , 2021, ACL.

[34]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[35]  Jeffrey Heer,et al.  Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models , 2021, ACL.

[36]  Peter Clark,et al.  Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering , 2018, EMNLP.

[37]  Ming-Wei Chang,et al.  Natural Questions: A Benchmark for Question Answering Research , 2019, TACL.

[38]  Peter Young,et al.  From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions , 2014, TACL.

[39]  Peter A. Flach,et al.  Explainability fact sheets: a framework for systematic assessment of explainable approaches , 2019, FAT*.

[40]  Christopher Potts,et al.  A large annotated corpus for learning natural language inference , 2015, EMNLP.

[41]  Ion Androutsopoulos,et al.  Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases , 2021, NAACL.

[42]  Tushar Khot,et al.  QASC: A Dataset for Question Answering via Sentence Composition , 2020, AAAI.

[43]  Yejin Choi,et al.  Edited Media Understanding: Reasoning About Implications of Manipulated Images , 2020, ArXiv.

[44]  Chenhao Tan,et al.  Evaluating and Characterizing Human Rationales , 2020, EMNLP.

[45]  Amina Adadi,et al.  Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) , 2018, IEEE Access.

[46]  Mohit Bansal,et al.  What Is More Likely to Happen Next? Video-and-Language Future Event Prediction , 2020, EMNLP.

[47]  Lyle Ungar,et al.  Does BERT Learn as Humans Perceive? Understanding Linguistic Styles through Lexica , 2021, EMNLP.

[48]  Sarah C. Kucker,et al.  An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results , 2019, Social Psychological and Personality Science.

[49]  Yejin Choi,et al.  Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning , 2019, EMNLP.

[50]  Harsh Jhamtani,et al.  Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering , 2020, EMNLP.

[51]  Georg Groh,et al.  Identifying and Measuring Annotator Bias Based on Annotators’ Demographic Characteristics , 2020, ALW.

[52]  Asim Kadav,et al.  Visual Entailment: A Novel Task for Fine-Grained Image Understanding , 2019, ArXiv.

[53]  Clayton T. Morrison,et al.  WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-hop Inference , 2018, LREC.

[54]  Dragomir R. Radev,et al.  ESPRIT: Explaining Solutions to Physical Reasoning Tasks , 2020, ACL.

[55]  Francisco Herrera,et al.  Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI , 2020, Inf. Fusion.

[56]  Albert Gatt,et al.  Grounded Textual Entailment , 2018, COLING.

[57]  N. Mostafazadeh,et al.  GLUCOSE: GeneraLized and COntextualized Story Explanations , 2020, Conference on Empirical Methods in Natural Language Processing.

[58]  Wang Ling,et al.  Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems , 2017, ACL.

[59]  Mark O. Riedl,et al.  Automated rationale generation: a technique for explainable AI and its effects on human perceptions , 2019, IUI.

[60]  Reut Tsarfaty,et al.  QADiscourse - Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines , 2020, EMNLP.

[61]  Emily M. Bender,et al.  Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science , 2018, TACL.

[62]  Ngoc Thang Vu,et al.  F1 Is Not Enough! Models and Evaluation towards User-Centered Explainable Question Answering , 2020, EMNLP.

[63]  Marcel van Gerven,et al.  Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges , 2018, ArXiv.

[64]  Timnit Gebru,et al.  Datasheets for datasets , 2018, Commun. ACM.

[65]  Yash Goyal,et al.  Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[66]  Jianfei Cai,et al.  VQA-E: Explaining, Elaborating, and Enhancing Your Answers for Visual Questions , 2018, ECCV.

[67]  Chris Welty,et al.  Crowd Truth: Harnessing disagreement in crowdsourcing a relation extraction gold standard , 2013 .

[68]  Colin Raffel,et al.  WT5?! Training Text-to-Text Models to Explain their Predictions , 2020, ArXiv.

[69]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[70]  Neema Kotonya,et al.  Explainable Automated Fact-Checking: A Survey , 2020, COLING.

[71]  Amit Dhurandhar,et al.  Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives , 2018, NeurIPS.

[72]  Danqi Chen,et al.  of the Association for Computational Linguistics: , 2001 .

[73]  Enrico Motta,et al.  An Ontology Design Pattern to Define Explanations , 2015, K-CAP.

[74]  Christopher Ré,et al.  Training Classifiers with Natural Language Explanations , 2018, ACL.

[75]  Regina Barzilay,et al.  Inferring Which Medical Treatments Work from Reports of Clinical Trials , 2019, NAACL.

[76]  Noah A. Smith,et al.  Evaluating Models’ Local Decision Boundaries via Contrast Sets , 2020, FINDINGS.

[77]  Yejin Choi,et al.  The Risk of Racial Bias in Hate Speech Detection , 2019, ACL.

[78]  Matthew Richardson,et al.  MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text , 2013, EMNLP.

[79]  Samuel R. Bowman,et al.  A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference , 2017, NAACL.

[80]  Trevor Darrell,et al.  Generating Counterfactual Explanations with Natural Language , 2018, ICML 2018.

[81]  Yi Yang,et al.  WikiQA: A Challenge Dataset for Open-Domain Question Answering , 2015, EMNLP.

[82]  Thomas Lukasiewicz,et al.  Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations , 2020, ACL.

[83]  Ana Marasovi'c,et al.  Explaining NLP Models via Minimal Contrastive Editing (MiCE) , 2021, FINDINGS.

[84]  Danqi Chen,et al.  CoQA: A Conversational Question Answering Challenge , 2018, TACL.

[85]  Shaohua Yang,et al.  Commonsense Justification for Action Explanation , 2018, EMNLP.

[86]  Trevor Darrell,et al.  Multimodal Explanations: Justifying Decisions and Pointing to the Evidence , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[87]  Regina Barzilay,et al.  Rationalizing Neural Predictions , 2016, EMNLP.

[88]  Xiang Ren,et al.  Teaching Machine Comprehension with Compositional Explanations , 2020, FINDINGS.

[89]  Tommi S. Jaakkola,et al.  Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control , 2019, EMNLP.

[90]  Eduard Hovy,et al.  Learning the Difference that Makes a Difference with Counterfactually-Augmented Data , 2020, ICLR.

[91]  Guokun Lai,et al.  RACE: Large-scale ReAding Comprehension Dataset From Examinations , 2017, EMNLP.

[92]  Masatoshi Tsuchiya,et al.  Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment , 2018, LREC.

[93]  Peter A. Jansen,et al.  WorldTree V2: A Corpus of Science-Domain Structured Explanations and Inference Patterns supporting Multi-Hop Inference , 2020, LREC.

[94]  Smaranda Muresan,et al.  Fact vs. Opinion: the Role of Argumentation Features in News Classification , 2020, COLING.

[95]  Yoav Goldberg,et al.  Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets , 2019, EMNLP.

[96]  Ingmar Weber,et al.  Racial Bias in Hate Speech and Abusive Language Detection Datasets , 2019, Proceedings of the Third Workshop on Abusive Language Online.

[97]  Yoshua Bengio,et al.  HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering , 2018, EMNLP.

[98]  Thomas Lukasiewicz,et al.  e-SNLI: Natural Language Inference with Natural Language Explanations , 2018, NeurIPS.

[99]  Praveen K. Paritosh,et al.  “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI , 2021, CHI.

[100]  Thomas Lukasiewicz,et al.  e-SNLI-VE: Corrected Visual-Textual Entailment with Natural Language Explanations , 2020, 2004.03744.

[101]  Zachary Chase Lipton.  The mythos of model interpretability , 2016, ACM Queue.

[102]  Tim Miller,et al.  Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[103]  Adina Williams,et al.  To what extent do human explanations of model behavior align with actual model behavior? , 2020, BLACKBOXNLP.

[104]  GiannottiFosca,et al.  A Survey of Methods for Explaining Black Box Models , 2018 .

[105]  John P. Dickerson,et al.  Counterfactual Explanations for Machine Learning: A Review , 2020, ArXiv.

[106]  Rachel Rudinger,et al.  Hypothesis Only Baselines in Natural Language Inference , 2018, *SEMEVAL.

[107]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[108]  J. Such,et al.  Explainable AI for Cultural Minds , 2020 .

[109]  Danqi Chen,et al.  Position-aware Attention and Supervised Data Improve Slot Filling , 2017, EMNLP.

[110]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[111]  Noah A. Smith,et al.  Measuring Association Between Labels and Free-Text Rationales , 2020, EMNLP.

[112]  Jeffrey Heer,et al.  Polyjuice: Automated, General-purpose Counterfactual Generation , 2021, ArXiv.

[113]  Brandon M. Greenwell,et al.  Interpretable Machine Learning , 2019, Hands-On Machine Learning with R.

[114]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[115]  Yejin Choi,et al.  Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervision , 2020, AAAI.

[116]  Yejin Choi,et al.  Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformation , 2021, ACL/IJCNLP.

[117]  Christine D. Piatko,et al.  Using “Annotator Rationales” to Improve Machine Learning for Text Categorization , 2007, NAACL.

[118]  Omer Levy,et al.  Annotation Artifacts in Natural Language Inference Data , 2018, NAACL.

[119]  Hongming Zhang,et al.  WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge , 2020, ACL.

[120]  Turking in the time of COVID , 2021, Behavior research methods.

[121]  Richard Socher,et al.  Explain Yourself! Leveraging Language Models for Commonsense Reasoning , 2019, ACL.

[122]  Or Biran,et al.  Explanation and Justification in Machine Learning : A Survey Or , 2017 .

[123]  Yoav Goldberg,et al.  Aligning Faithful Interpretations with their Social Attribution , 2020, ArXiv.

[124]  Hannaneh Hajishirzi,et al.  Fact or Fiction: Verifying Scientific Claims , 2020, EMNLP.

[125]  Jonathan Berant,et al.  Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies , 2021, Transactions of the Association for Computational Linguistics.

[126]  Regina Barzilay,et al.  Deriving Machine Attention from Human Rationales , 2018, EMNLP.

[127]  Jun Yan,et al.  Learning from Explanations with Neural Execution Tree , 2020, ICLR.

[128]  Gary Klein,et al.  Metrics for Explainable AI: Challenges and Prospects , 2018, ArXiv.

[129]  Andreas Vlachos,et al.  FEVER: a Large-scale Dataset for Fact Extraction and VERification , 2018, NAACL.

[130]  André F. T. Martins,et al.  Do Context-Aware Translation Models Pay the Right Attention? , 2021, ACL.

[131]  Yejin Choi,et al.  Social Chemistry 101: Learning to Reason about Social and Moral Norms , 2020, EMNLP.

[132]  Ye Zhang,et al.  Do Human Rationales Improve Machine Explanations? , 2019, BlackboxNLP@ACL.

[133]  Jure Leskovec,et al.  Learning Attitudes and Attributes from Multi-aspect Reviews , 2012, 2012 IEEE 12th International Conference on Data Mining.

[134]  Yue Zhang,et al.  Does it Make Sense? And Why? A Pilot Study for Sense Making and Explanation , 2019, ACL.

[135]  Marco F. Huber,et al.  A Survey on the Explainability of Supervised Machine Learning , 2020, J. Artif. Intell. Res..

[136]  Yuval Pinter,et al.  Attention is not not Explanation , 2019, EMNLP.

[137]  Mucahid Kutlu,et al.  Annotator Rationales for Labeling Tasks in Crowdsourcing , 2020, J. Artif. Intell. Res..

[138]  D. Hilton.  Conversational processes and causal explanation. , 1990 .

[139]  Byron C. Wallace,et al.  ERASER: A Benchmark to Evaluate Rationalized NLP Models , 2020, ACL.

[140]  Trevor Darrell,et al.  Textual Explanations for Self-Driving Vehicles , 2018, ECCV.

[141]  Lalana Kagal,et al.  Explaining Explanations: An Overview of Interpretability of Machine Learning , 2018, 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA).

[142]  Peter Jansen,et al.  What’s in an Explanation? Characterizing Knowledge and Inference Requirements for Elementary Science Exams , 2016, COLING.

[143]  Ali Farhadi,et al.  From Recognition to Cognition: Visual Commonsense Reasoning , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).