How to Extract More Information with Less Burden: Fundus Image Classification and Retinal Disease Localization with Ophthalmologist Intervention

Image classification using deep convolutional neural networks (DCNN) has a competitive performance as compare to other state-of-the-art methods. Here, attention can be visualized as a heatmap to improve the explainability of DCNN. We generated the initial heatmaps by using gradient-based classification activation map (Grad-CAM). We firstly assume that these Grad-CAM heatmaps can reveal the lesion regions well, then apply the attention mining on these heatmaps. Another, we assume that these Grad-CAM heatmaps can't reveal the lesion regions well then apply the dissimilarity loss on these Grad-CAM heatmaps. In this study, we asked the ophthalmologists to select 30% of the heatmaps. Furthermore, we design a knowledge preservation (KP) loss to minimize the discrepancy between heatmaps generated from the updated network and the selected heatmaps. Experiments revealed that our method improved accuracy from 90.1% to 96.2%. We also found that the attention regions are closer to the GT lesion regions.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Yun Fu,et al.  Tell Me Where to Look: Guided Attention Inference Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  M. Usman Akram,et al.  Drusen exudate lesion discrimination in colour fundus images , 2014, 2014 14th International Conference on Hybrid Intelligent Systems.

[4]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[5]  Trevor Darrell,et al.  Constrained Convolutional Neural Networks for Weakly Supervised Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[6]  Gernot A. Fink,et al.  Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[7]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  C. Dolea,et al.  World Health Organization , 1949, International Organization.

[9]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[10]  Chengdong Wu,et al.  Automatic Fovea Center Localization in Retinal Images Using Saliency-Guided Object Discovery and Feature Extraction , 2017 .

[11]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[12]  Hiroshi Fujita,et al.  Automated detection of retinal nerve fiber layer defects on fundus images: false positive reduction based on vessel likelihood , 2016, SPIE Medical Imaging.

[13]  Anurag Mittal,et al.  Automated feature extraction for early detection of diabetic retinopathy in fundus images , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.