Explain Any Concept: Segment Anything Meets Concept-Based Explanation

EXplainable AI (XAI) is an essential topic to improve human understanding of deep neural networks (DNNs) given their black-box internals. For computer vision tasks, mainstream pixel-based XAI methods explain DNN decisions by identifying important pixels, and emerging concept-based XAI explore forming explanations with concepts (e.g., a head in an image). However, pixels are generally hard to interpret and sensitive to the imprecision of XAI methods, whereas"concepts"in prior works require human annotation or are limited to pre-defined concept sets. On the other hand, driven by large-scale pre-training, Segment Anything Model (SAM) has been demonstrated as a powerful and promotable framework for performing precise and comprehensive instance segmentation, enabling automatic preparation of concept sets from a given image. This paper for the first time explores using SAM to augment concept-based XAI. We offer an effective and flexible concept-based explanation method, namely Explain Any Concept (EAC), which explains DNN decisions with any concept. While SAM is highly effective and offers an"out-of-the-box"instance segmentation, it is costly when being integrated into defacto XAI pipelines. We thus propose a lightweight per-input equivalent (PIE) scheme, enabling efficient explanation with a surrogate model. Our evaluation over two popular datasets (ImageNet and COCO) illustrate the highly encouraging performance of EAC over commonly-used XAI methods.

[1]  L. Herm.  Impact Of Explainable AI On Cognitive Load: Insights From An Empirical Study , 2023, ECIS.

[2]  V. Mezaris,et al.  TAME: Attention Mechanism Based Feature Fusion for Generating Explanation Maps of Convolutional Neural Networks , 2022, 2022 IEEE International Symposium on Multimedia (ISM).

[3]  Qi Pang,et al.  Unveiling Hidden DNN Defects with Decision-Based Metamorphic Testing , 2022, ASE.

[4]  V. Mezaris,et al.  Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanism , 2022, ECCV Workshops.

[5]  Thomas Serre,et al.  What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods , 2021, NeurIPS.

[6]  Kun Kuang,et al.  Instance-wise or Class-wise? A Tale of Neighbor Shapley for Concept-based Explanation , 2021, ACM Multimedia.

[7]  Caleb Chen Cao,et al.  Roadmap of Designing Cognitive Metrics for Explainable Artificial Intelligence (XAI) , 2021, ArXiv.

[8]  Su-In Lee,et al.  Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regression , 2020, AISTATS.

[9]  Xixian Chen,et al.  Towards Global Explanations of Convolutional Neural Networks With Concept Attribution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Adrian Weller,et al.  Evaluating and Aggregating Feature-based Model Explanations , 2020, IJCAI.

[11]  Hugh Chen,et al.  From local explanations to global understanding with explainable AI for trees , 2020, Nature Machine Intelligence.

[12]  Sercan Ö. Arik,et al.  On Completeness-aware Concept-Based Explanations in Deep Neural Networks , 2019, NeurIPS.

[13]  Chun-Liang Li,et al.  On Concept-Based Explanations in Deep Neural Networks , 2019, ArXiv.

[14]  Yash Goyal,et al.  Explaining Classifiers with Causal Concept Effect (CaCE) , 2019, ArXiv.

[15]  Markus H. Gross,et al.  Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation , 2019, ICML.

[16]  Klaus Mueller,et al.  Beyond saliency: understanding convolutional neural networks from saliency prediction on layer-wise relevance propagation , 2017, Image Vis. Comput..

[17]  Amina Adadi,et al.  Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) , 2018, IEEE Access.

[18]  Kate Saenko,et al.  RISE: Randomized Input Sampling for Explanation of Black-box Models , 2018, BMVC.

[19]  Martin Wattenberg,et al.  Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) , 2017, ICML.

[20]  Cengiz Öztireli,et al.  Towards better understanding of gradient-based attribution methods for Deep Neural Networks , 2017, ICLR.

[21]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[22]  Andrea Vedaldi,et al.  Interpretable Explanations of Black Boxes by Meaningful Perturbation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  Avanti Shrikumar,et al.  Learning Important Features Through Propagating Activation Differences , 2017, ICML.

[24]  Ankur Taly,et al.  Axiomatic Attribution for Deep Networks , 2017, ICML.

[25]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Alexander Binder,et al.  Evaluating the Visualization of What a Deep Neural Network Has Learned , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Barry L. Nelson,et al.  Shapley Effects for Global Sensitivity Analysis: Theory and Computation , 2016, SIAM/ASA J. Uncertain. Quantification.

[28]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[29]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Andrea Vedaldi,et al.  Visualizing Deep Convolutional Neural Networks Using Natural Pre-images , 2015, International Journal of Computer Vision.

[32]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[35]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[36]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  L. Shapley.  A Value for n-person Games , 1988 .