XAI Method Properties: A (Meta-)study

In the meantime, a wide variety of terminologies, motivations, approaches and evaluation criteria have been developed within the scope of research on explainable artificial intelligence (XAI). Many taxonomies can be found in the literature, each with a different focus, but also showing many points of overlap. In this paper, we summarize the most cited and current taxonomies in a meta-analysis in order to highlight the essential aspects of the state-of-the-art in XAI. We also present and add terminologies as well as concepts from a large number of survey articles on the topic. Last but not least, we illustrate concepts from the higher-level taxonomy with more than 50 example methods, which we categorize accordingly, thus providing a wide-ranging overview of aspects of XAI and paving the way for use case-appropriate as well as context-specific subsequent research.

[1]  Andrew Cropper,et al.  Turning 30: New Ideas in Inductive Logic Programming , 2020, ArXiv.

[2]  Jude W. Shavlik,et al.  in Advances in Neural Information Processing , 1996 .

[3]  Ankur Taly,et al.  Axiomatic Attribution for Deep Networks , 2017, ICML.

[4]  Cuntai Guan,et al.  A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[6]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[7]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[8]  Sarah Adel Bargal,et al.  NBDT: Neural-Backed Decision Trees , 2020, ArXiv.

[9]  Jianlong Zhou,et al.  Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics , 2021, Electronics.

[10]  Klaus-Robert Müller,et al.  Learning how to explain neural networks: PatternNet and PatternAttribution , 2017, ICLR.

[11]  Motoaki Kawanabe,et al.  How to Explain Individual Classification Decisions , 2009, J. Mach. Learn. Res..

[12]  Quanshi Zhang,et al.  Visual interpretability for deep learning: a survey , 2018, Frontiers of Information Technology & Electronic Engineering.

[13]  Stephen Muggleton,et al.  Ultra-Strong Machine Learning: comprehensibility of programs learned with ILP , 2018, Machine Learning.

[14]  Xue Liu,et al.  An Empirical Evaluation of Rule Extraction from Recurrent Neural Networks , 2017, Neural Computation.

[15]  Natalia Díaz Rodríguez,et al.  Explainability in Deep Reinforcement Learning , 2020, Knowl. Based Syst..

[16]  Martin Wattenberg,et al.  SmoothGrad: removing noise by adding noise , 2017, ArXiv.

[17]  Mennatallah El-Assady,et al.  explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning , 2019, IEEE Transactions on Visualization and Computer Graphics.

[18]  Joseph D. Janizek,et al.  Explaining Explanations: Axiomatic Feature Interactions for Deep Networks , 2020, J. Mach. Learn. Res..

[19]  Quanshi Zhang,et al.  Interpreting CNN knowledge via an Explanatory Graph , 2017, AAAI.

[20]  Holger Hermanns,et al.  What Do We Want From Explainable Artificial Intelligence (XAI)? - A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research , 2021, Artif. Intell..

[21]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[22]  Naftali Tishby,et al.  Opening the Black Box of Deep Neural Networks via Information , 2017, ArXiv.

[23]  Satya M. Muddamsetty,et al.  Introducing and assessing the explainable AI (XAI)method: SIDU , 2021 .

[24]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[25]  Stephen Muggleton,et al.  How Does Predicate Invention Affect Human Comprehensibility? , 2016, ILP.

[26]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Brandon M. Greenwell,et al.  Interpretable Machine Learning , 2019, Hands-On Machine Learning with R.

[28]  Eneldo Loza Mencía,et al.  DeepRED - Rule Extraction from Deep Neural Networks , 2016, DS.

[29]  Franco Turini,et al.  A Survey of Methods for Explaining Black Box Models , 2018, ACM Comput. Surv..

[30]  Dong Tian,et al.  FoldingNet: Interpretable Unsupervised Learning on 3D Point Clouds , 2017, ArXiv.

[31]  Amit Dhurandhar,et al.  One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques , 2019, ArXiv.

[32]  Lalana Kagal,et al.  Explaining Explanations: An Overview of Interpretability of Machine Learning , 2018, 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA).

[33]  Klaus-Robert Müller,et al.  iNNvestigate neural networks! , 2018, J. Mach. Learn. Res..

[34]  Trevor Darrell,et al.  Grounding Visual Explanations , 2018, ECCV.

[35]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[36]  Bolei Zhou,et al.  Interpretable Basis Decomposition for Visual Explanation , 2018, ECCV.

[37]  Kate Saenko,et al.  RISE: Randomized Input Sampling for Explanation of Black-box Models , 2018, BMVC.

[38]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[39]  Jaime S. Cardoso,et al.  Machine Learning Interpretability: A Survey on Methods and Metrics , 2019, Electronics.

[40]  Ute Schmid,et al.  Expressive Explanations of DNNs by Combining Concept Analysis with ILP , 2020, KI.

[41]  Trevor Darrell,et al.  Textual Explanations for Self-Driving Vehicles , 2018, ECCV.

[42]  Alexander Binder,et al.  Unmasking Clever Hans predictors and assessing what machines really learn , 2019, Nature Communications.

[43]  Amina Adadi,et al.  Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) , 2018, IEEE Access.

[44]  L. Longo,et al.  Explainable Artificial Intelligence: a Systematic Review , 2020, ArXiv.

[45]  William J. Clancey,et al.  Principles of Explanation in Human-AI Systems , 2021, ArXiv.

[46]  Amit Dhurandhar,et al.  Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives , 2018, NeurIPS.

[47]  Cynthia Rudin,et al.  This Looks Like That: Deep Learning for Interpretable Image Recognition , 2018 .

[48]  Kun Qian,et al.  A Survey of the State of Explainable AI for Natural Language Processing , 2020, AACL/IJCNLP.

[49]  Gereon Weiss,et al.  Benchmarking Uncertainty Estimation Methods for Deep Learning With Safety-Related Metrics , 2020, SafeAI@AAAI.

[50]  Klaus-Robert Müller,et al.  From Clustering to Cluster Explanations via Neural Networks , 2019, IEEE transactions on neural networks and learning systems.

[51]  Roberto Cipolla,et al.  Concrete Problems for Autonomous Vehicle Safety: Advantages of Bayesian Deep Learning , 2017, IJCAI.

[52]  Anind K. Dey,et al.  Understanding and Using Context , 2001, Personal and Ubiquitous Computing.

[53]  Przemyslaw Biecek,et al.  The grammar of interactive explanatory model analysis , 2020, Data mining and knowledge discovery.

[54]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[55]  Ute Schmid,et al.  The Next Generation of Medical Decision Support: A Roadmap Toward Transparent Expert Companions , 2020, Frontiers in Artificial Intelligence.

[56]  Guang-Zhong Yang,et al.  XAI—Explainable artificial intelligence , 2019, Science Robotics.

[57]  Applying Explainable Artificial Intelligence for Deep Learning Networks to Decode Facial Expressions of Pain and Emotions , 2018 .

[58]  Michael Chromik,et al.  A Taxonomy for Human Subject Evaluation of Black-Box Explanations in XAI , 2020, ExSS-ATEC@IUI.

[59]  Martin Wattenberg,et al.  Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) , 2017, ICML.

[60]  Michael Siebers,et al.  Explaining Black-Box Classifiers with ILP - Empowering LIME with Aleph to Approximate Non-linear Decisions with Relational Rules , 2018, ILP.

[61]  Rich Caruana,et al.  How Interpretable and Trustworthy are GAMs? , 2020, KDD.

[62]  Quanshi Zhang,et al.  Explaining Neural Networks Semantically and Quantitatively , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[63]  Kate Saenko,et al.  Black-box Explanation of Object Detectors via Saliency Maps , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  C. Rudin,et al.  Concept whitening for interpretable image recognition , 2020, Nature Machine Intelligence.

[65]  Andrea Vedaldi,et al.  Net2Vec: Quantifying and Explaining How Concepts are Encoded by Filters in Deep Neural Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[66]  William Eberle,et al.  Explainable Artificial Intelligence Approaches: A Survey , 2021, ArXiv.

[67]  Jingtao Yao.  Knowledge extracted from trained neural networks: What's next? , 2005, SPIE Defense + Commercial Sensing.

[68]  Rich Caruana,et al.  InterpretML: A Unified Framework for Machine Learning Interpretability , 2019, ArXiv.

[69]  Eric M. S. P. Veith,et al.  Explainable Reinforcement Learning: A Survey , 2020, CD-MAKE.

[70]  Bjorn Ommer,et al.  A Disentangling Invertible Interpretation Network for Explaining Latent Representations , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[71]  Percy Liang,et al.  Understanding Black-box Predictions via Influence Functions , 2017, ICML.

[72]  Lee Lacy,et al.  Defense Advanced Research Projects Agency (DARPA) Agent Markup Language Computer Aided Knowledge Acquisition , 2005 .

[73]  John F. Canny,et al.  Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[74]  Trevor Darrell,et al.  Generating Visual Explanations , 2016, ECCV.

[75]  Claes Lundström,et al.  Survey of XAI in Digital Pathology , 2020, AI and ML for Digital Pathology.

[76]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[77]  Kristian Kersting,et al.  Explanatory Interactive Machine Learning , 2019, AIES.

[78]  Juliana Jansen Ferreira,et al.  What Are People Doing About XAI User Experience? A Survey on AI Explainability Research and Practice , 2020, HCI.

[79]  Zachary Chase Lipton.  The mythos of model interpretability , 2016, ACM Queue.

[80]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[81]  Weng-Keen Wong,et al.  Explanatory Debugging: Supporting End-User Debugging of Machine-Learned Programs , 2010, VL/HCC.

[82]  Seth Flaxman,et al.  European Union Regulations on Algorithmic Decision-Making and a "Right to Explanation" , 2016, AI Mag..

[83]  Bernt Schiele,et al.  Interpretability Beyond Classification Output: Semantic Bottleneck Networks , 2019, ArXiv.

[84]  Bolei Zhou,et al.  Network Dissection: Quantifying Interpretability of Deep Visual Representations , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[85]  Jerry Alan Fails,et al.  Interactive machine learning , 2003, IUI '03.

[86]  Dietmar Jannach,et al.  A systematic review and taxonomy of explanations in decision support and recommender systems , 2017, User Modeling and User-Adapted Interaction.

[87]  Wei Bai,et al.  Quantitative Evaluations on Saliency Methods: An Experimental Study , 2020, ArXiv.

[88]  Artur S. d'Avila Garcez,et al.  Logic Tensor Networks for Semantic Image Interpretation , 2017, IJCAI.

[89]  Xu Chen,et al.  Explainable Recommendation: A Survey and New Perspectives , 2018, Found. Trends Inf. Retr..

[90]  Emil Pitkin,et al.  Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation , 2013, 1309.6392.

[91]  Klaus-Robert Müller,et al.  Towards Explainable Artificial Intelligence , 2019, Explainable AI.

[92]  Amitojdeep Singh,et al.  Explainable Deep Learning Models in Medical Image Analysis , 2020, J. Imaging.

[93]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[94]  T. Kathirvalavakumar,et al.  Rule extraction from neural networks — A comparative study , 2012, International Conference on Pattern Recognition, Informatics and Medical Engineering (PRIME-2012).

[95]  Sebastian Thrun,et al.  Extracting Rules from Artifical Neural Networks with Distributed Representations , 1994, NIPS.

[96]  Deborah Silver,et al.  Feature Visualization , 1994, Scientific Visualization.

[97]  William J. Clancey,et al.  Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI , 2019, ArXiv.

[98]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[99]  Adel Rahimi,et al.  The Need for Standardized Explainability , 2020, ArXiv.

[100]  Marcin Detyniecki,et al.  Concept Tree: High-Level Representation of Variables for More Interpretable Surrogate Decision Trees , 2019, ArXiv.

[101]  Wojciech Samek,et al.  Toward Interpretable Machine Learning: Transparent Deep Neural Networks and Beyond , 2020, ArXiv.

[102]  Tameru Hailesilassie,et al.  Rule Extraction Algorithm for Deep Neural Networks: A Review , 2016, ArXiv.

[103]  Bettina Finzel,et al.  Mutual Explanations for Cooperative Decision Making in Medicine , 2020, KI - Künstliche Intelligenz.

[104]  Andrea Omicini,et al.  On the integration of symbolic and sub-symbolic techniques for XAI: A survey , 2020, Intelligenza Artificiale.

[105]  Xue Liu,et al.  A Comparative Study of Rule Extraction for Recurrent Neural Networks , 2018, 1801.05420.

[106]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[107]  Andrea Vedaldi,et al.  Interpretable Explanations of Black Boxes by Meaningful Perturbation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[108]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[109]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[110]  James Zou,et al.  Towards Automatic Concept-based Explanations , 2019, NeurIPS.

[111]  Hu Wang,et al.  ReNN: Rule-embedded Neural Networks , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[112]  Marcel van Gerven,et al.  Explainable Deep Learning: A Field Guide for the Uninitiated , 2020, J. Artif. Intell. Res..

[113]  Sotiris Kotsiantis,et al.  Explainable AI: A Review of Machine Learning Interpretability Methods , 2020, Entropy.

[114]  Chun-Liang Li,et al.  On Completeness-aware Concept-Based Explanations in Deep Neural Networks , 2020, NeurIPS.

[115]  Francisco Herrera,et al.  Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI , 2020, Inf. Fusion.

[116]  Christian Biemann,et al.  What do we need to build explainable AI systems for the medical domain? , 2017, ArXiv.