Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks

To verify and validate networks, it is essential to gain insight into their decisions, limitations as well as possible shortcomings of training data. In this work, we propose a post-hoc, optimization based visual explanation method, which highlights the evidence in the input image for a specific prediction. Our approach is based on a novel technique to defend against adversarial evidence (i.e. faulty evidence due to artefacts) by filtering gradients during optimization. The defense does not depend on human-tuned parameters. It enables explanations which are both fine-grained and preserve the characteristics of images, such as edges and colors. The explanations are interpretable, suited for visualizing detailed evidence and can be tested as they are valid model inputs. We qualitatively and quantitatively evaluate our approach on a multitude of models and datasets.

[1]  E. Finkelstein,et al.  Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes , 2017, JAMA.

[2]  Peng Liu,et al.  Uniqueness-Driven Saliency Analysis for Automated Lesion Detection with Applications to Retinal Diseases , 2018, MICCAI.

[3]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[5]  Sameer Singh,et al.  “Why Should I Trust You?”: Explaining the Predictions of Any Classifier , 2016, NAACL.

[6]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[7]  R. Klein,et al.  Global Prevalence and Major Risk Factors of Diabetic Retinopathy , 2012, Diabetes Care.

[8]  Vineeth N. Balasubramanian,et al.  Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[9]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[10]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[11]  Gernot A. Fink,et al.  Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[12]  Jie Yang,et al.  Automatic hemorrhage detection in color fundus images based on gradual removal of vascular branches , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[13]  Yarin Gal,et al.  Real Time Image Saliency for Black Box Classifiers , 2017, NIPS.

[14]  Quanshi Zhang,et al.  Visual interpretability for deep learning: a survey , 2018, Frontiers of Information Technology & Electronic Engineering.

[15]  Avanti Shrikumar,et al.  Learning Important Features Through Propagating Activation Differences , 2017, ICML.

[16]  Il-Seok Oh,et al.  Regional Multi-Scale Approach for Visually Pleasing Explanations of Deep Neural Networks , 2018, IEEE Access.

[17]  Motoaki Kawanabe,et al.  How to Explain Individual Classification Decisions , 2009, J. Mach. Learn. Res..

[18]  Qingquan Song,et al.  Towards Explanation of DNN-based Prediction with Guided Feature Inversion , 2018, KDD.

[19]  Kate Saenko,et al.  RISE: Randomized Input Sampling for Explanation of Black-box Models , 2018, BMVC.

[20]  B. Klein,et al.  Global Prevalence and Major Risk Factors of Diabetic Retinopathy , 2012, Diabetes Care.

[21]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[22]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[24]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[25]  Xia Hu,et al.  Techniques for interpretable machine learning , 2018, Commun. ACM.

[26]  Andrea Vedaldi,et al.  Interpretable Explanations of Black Boxes by Meaningful Perturbation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[28]  Vijay M. Mane,et al.  Detection of Red lesions in diabetic retinopathy affected fundus images , 2015, 2015 IEEE International Advance Computing Conference (IACC).

[29]  Roberto Cipolla,et al.  Concrete Problems for Autonomous Vehicle Safety: Advantages of Bayesian Deep Learning , 2017, IJCAI.

[30]  Jorge A Cuadros,et al.  EyePACS: An Adaptable Telemedicine System for Diabetic Retinopathy Screening , 2009, Journal of diabetes science and technology.

[31]  Arunkumar Rajendran,et al.  Multi-retinal disease classification by reduced deep learning features , 2017, Neural Computing and Applications.

[32]  Martin Wattenberg,et al.  SmoothGrad: removing noise by adding noise , 2017, ArXiv.

[33]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Samarendra Dandapat,et al.  A Gaussian Scale Space Approach For Exudates Detection, Classification And Severity Prediction , 2015, ArXiv.

[35]  H. Kälviäinen,et al.  DIARETDB 1 diabetic retinopathy database and evaluation protocol , 2007 .

[36]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[37]  Yang Zhang,et al.  A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations , 2018, ICML.

[38]  A. S. Jalal,et al.  A survey on automated microaneurysm detection in Diabetic Retinopathy retinal images , 2013, 2013 International Conference on Information Systems and Computer Networks.

[39]  Jennifer K. Sun,et al.  Diabetic Retinopathy: A Position Statement by the American Diabetes Association , 2017, Diabetes Care.

[40]  Samy Bengio,et al.  Adversarial examples in the physical world , 2016, ICLR.

[41]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[42]  Song Han,et al.  Learning both Weights and Connections for Efficient Neural Network , 2015, NIPS.

[43]  Zhe L. Lin,et al.  Top-Down Neural Attention by Excitation Backprop , 2016, International Journal of Computer Vision.

[44]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Joni-Kristian Kämäräinen,et al.  The DIARETDB1 Diabetic Retinopathy Database and Evaluation Protocol , 2007, BMVC.

[46]  Jie Chen,et al.  A location-to-segmentation strategy for automatic exudate segmentation in colour retinal fundus images , 2017, Comput. Medical Imaging Graph..

[47]  B. Schmauch,et al.  Deep learning approach for diabetic retinopathy screening , 2016 .

[48]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[49]  Bolei Zhou,et al.  Object Detectors Emerge in Deep Scene CNNs , 2014, ICLR.

[50]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[51]  David Duvenaud,et al.  Explaining Image Classifiers by Counterfactual Generation , 2018, ICLR.

[52]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[53]  Ankur Taly,et al.  Axiomatic Attribution for Deep Networks , 2017, ICML.

[54]  Been Kim,et al.  Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values , 2018, ICLR.