Extracting Causal Visual Features For Limited Label Classification

Neural networks trained to classify images do so by identifying features that allow them to distinguish between classes. These sets of features are either causal or context dependent. Grad-CAM is a popular method of visualizing both sets of features. In this paper, we formalize this feature divide and provide a methodology to extract causal features from Grad-CAM. We do so by defining context features as those features that allow contrast between predicted class and any contrast class. We then apply a set theoretic approach to separate causal from contrast features for COVID-19 CT scans. We show that on average, the image regions with the proposed causal features require 15% less bits when encoded using Huffman encoding, compared to Grad-CAM, for an average increase of 3% classification accuracy, over Grad-CAM. Moreover, we validate the transfer-ability of causal features between networks and comment on the non-human interpretable causal nature of current networks.

[1]  Vineeth N. Balasubramanian,et al.  Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[2]  David A. Huffman,et al.  A method for the construction of minimum-redundancy codes , 1952, Proceedings of the IRE.

[3]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Ghassan Al-Regib,et al.  A Machine Learning Benchmark for Facies Classification , 2019, Interpretation.

[6]  Ghassan Al-Regib,et al.  Distorted Representation Space Characterization Through Backpropagated Gradients , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[7]  Ghassan Al-Regib,et al.  Relative Afferent Pupillary Defect Screening Through Transfer Learning , 2019, IEEE Journal of Biomedical and Health Informatics.

[8]  László E. Szabó,et al.  On Reichenbach's Common Cause Principle and Reichenbach's Notion of Common Cause , 1998, The British Journal for the Philosophy of Science.

[9]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Ghassan AlRegib,et al.  Gradients as a Measure of Uncertainty in Neural Networks , 2020, 2020 IEEE International Conference on Image Processing (ICIP).

[11]  Ghassan AlRegib,et al.  Implicit Saliency In Deep Neural Networks , 2020, 2020 IEEE International Conference on Image Processing (ICIP).

[12]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[13]  Ghassan AlRegib,et al.  Towards understanding common features between natural and seismic images , 2018, SEG Technical Program Expanded Abstracts 2018.

[14]  Bernhard Schölkopf,et al.  Discovering Causal Signals in Images , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Kate Saenko,et al.  RISE: Randomized Input Sampling for Explanation of Black-box Models , 2018, BMVC.

[16]  Joshua B. Tenenbaum,et al.  Inferring causal networks from observations and interventions , 2003, Cogn. Sci..

[17]  P. Xie,et al.  COVID-CT-Dataset: A CT Scan Dataset about COVID-19 , 2020, ArXiv.

[18]  Ghassan AlRegib,et al.  Contrastive Explanations In Neural Networks , 2020, 2020 IEEE International Conference on Image Processing (ICIP).

[19]  Ghassan AlRegib,et al.  Backpropagated Gradient Representations for Anomaly Detection , 2020, ECCV.

[20]  A. Torralba,et al.  The role of context in object recognition , 2007, Trends in Cognitive Sciences.

[21]  X. He,et al.  Sample-Efficient Deep Learning for COVID-19 Diagnosis Based on CT Scans , 2020, medRxiv.