Explainable end-to-end deep learning for diabetic retinopathy detection across multiple datasets

Abstract. Purpose: Diabetic retinopathy (DR) is characterized by retinal lesions affecting people having diabetes for several years. It is one of the leading causes of visual impairment worldwide. To diagnose this disease, ophthalmologists need to manually analyze retinal fundus images. Computer-aided diagnosis systems can help alleviate this burden by automatically detecting DR on retinal images, thus saving physicians’ precious time and reducing costs. The objective of this study is to develop a deep learning algorithm capable of detecting DR on retinal fundus images. Nine public datasets and more than 90,000 images are used to assess the efficiency of the proposed technique. In addition, an explainability algorithm is developed to visually show the DR signs detected by the deep model. Approach: The proposed deep learning algorithm fine-tunes a pretrained deep convolutional neural network for DR detection. The model is trained on a subset of EyePACS dataset using a cosine annealing strategy for decaying the learning rate with warm up, thus improving the training accuracy. Tests are conducted on the nine datasets. An explainability algorithm based on gradient-weighted class activation mapping is developed to visually show the signs selected by the model to classify the retina images as DR. Result: The proposed network leads to higher classification rates with an area under curve (AUC) of 0.986, sensitivity = 0.958, and specificity = 0.971 for EyePACS. For MESSIDOR, MESSIDOR-2, DIARETDB0, DIARETDB1, STARE, IDRID, E-ophtha, and UoA-DR, the AUC is 0.963, 0.979, 0.986, 0.988, 0.964, 0.957, 0.984, and 0.990, respectively. Conclusions: The obtained results achieve state-of-the-art performance and outperform past published works relying on training using only publicly available datasets. The proposed approach can robustly classify fundus images and detect DR. An explainability model was developed and showed that our model was able to efficiently identify different signs of DR and detect this health issue.

[1]  Manually segmented vascular networks from images of retina with proliferative diabetic and hypertensive retinopathy , 2018, Data in brief.

[2]  Mubarak Shah,et al.  Norm-Preservation: Why Residual Networks Can Become Extremely Deep? , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Frans Coenen,et al.  Convolutional Neural Networks for Diabetic Retinopathy , 2016, MIUA.

[4]  James M. Brown,et al.  Automated Diagnosis of Plus Disease in Retinopathy of Prematurity Using Deep Convolutional Neural Networks , 2018, JAMA ophthalmology.

[5]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[6]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[7]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[8]  Mohamad Ivan Fanany,et al.  Residual convolutional neural network for diabetic retinopathy , 2017, 2017 International Conference on Advanced Computer Science and Information Systems (ICACSIS).

[9]  Akshya Swain,et al.  An Efficient Framework for Automated Screening of Clinically Significant Macular Edema , 2020, Comput. Biol. Medicine.

[10]  Di Xiao,et al.  Exudate detection for diabetic retinopathy with convolutional neural networks , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[11]  Zhi Zhang,et al.  Bag of Tricks for Image Classification with Convolutional Neural Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Renoh Johnson Chalakkal,et al.  Quality and content analysis of fundus images using deep learning , 2019, Comput. Biol. Medicine.

[13]  V. Sudha,et al.  Diabetic Retinopathy Detection , 2020, International Journal of Engineering and Advanced Technology.

[14]  Kemal Adem,et al.  Exudate detection for diabetic retinopathy with circular Hough transformation and convolutional neural networks , 2018, Expert Syst. Appl..

[15]  Yuan Luo,et al.  Automated Detection of Diabetic Retinopathy using a Binocular Siamese-Like Convolutional Network , 2019, 2019 IEEE International Symposium on Circuits and Systems (ISCAS).

[16]  Yung-Hui Li,et al.  Computer-Assisted Diagnosis for Diabetic Retinopathy Based on Fundus Images Using Deep Convolutional Neural Network , 2019, Mob. Inf. Syst..

[17]  Laude,et al.  FEEDBACK ON A PUBLICLY DISTRIBUTED IMAGE DATABASE: THE MESSIDOR DATABASE , 2014 .

[18]  Kaiming He,et al.  Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour , 2017, ArXiv.

[19]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Gwénolé Quellec,et al.  Deep image mining for diabetic retinopathy screening , 2016, Medical Image Anal..

[21]  Prachi Gharpure,et al.  Diabetic retinopathy detection using deep convolutional neural networks , 2016, 2016 International Conference on Computing, Analytics and Security Trends (CAST).

[22]  Dennis Wollersheim,et al.  Pulmonary nodule classification with deep residual networks , 2017, International Journal of Computer Assisted Radiology and Surgery.

[23]  Bram van Ginneken,et al.  Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images , 2016, IEEE Transactions on Medical Imaging.

[24]  Xiaogang Wang,et al.  Zoom-in-Net: Deep Mining Lesions for Diabetic Retinopathy Detection , 2017, MICCAI.

[25]  Gernot A. Fink,et al.  Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[26]  Matthew B. Blaschko,et al.  An ensemble deep learning based approach for red lesion detection in fundus images , 2017, Comput. Methods Programs Biomed..

[27]  Romany F Mansour,et al.  Deep-learning-based automatic computer-aided diagnosis system for diabetic retinopathy , 2017, Biomedical Engineering Letters.

[28]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[29]  Waleed H. Abdulla,et al.  Comparative Analysis of University of Auckland Diabetic Retinopathy Database , 2017, ICSPS 2017.

[30]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[31]  Lars Ailo Bongo,et al.  Reproduction study using public data of: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs , 2018, PloS one.

[32]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Chia-Hung Yeh,et al.  Transforming Retinal Photographs to Entropy Images in Deep Learning to Improve Automated Detection for Diabetic Retinopathy , 2018, Journal of ophthalmology.

[34]  B. Schmauch,et al.  Deep learning approach for diabetic retinopathy screening , 2016 .

[35]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[36]  T. Vinding,et al.  Prevalence and causes of visual impairment according to World Health Organization and United States criteria in an aged, urban Scandinavian population: the Copenhagen City Eye Study. , 2001, Ophthalmology.

[37]  M. Saquib Sarfraz,et al.  Head Pose Estimation in Face Recognition Across Pose Scenarios , 2008, VISAPP.

[38]  Rishab Gargeya,et al.  Automated Identification of Diabetic Retinopathy Using Deep Learning. , 2017, Ophthalmology.

[39]  Dwarikanath Mahapatra,et al.  A novel hybrid approach for severity assessment of Diabetic Retinopathy in colour fundus images , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[40]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[41]  Moustapha Kardouchi,et al.  Diabetic Retinopathy Detection Using Machine Learning and Texture Features , 2018, 2018 IEEE Canadian Conference on Electrical & Computer Engineering (CCECE).

[42]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[43]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[44]  Junhao Wen,et al.  Exudate Detection for Diabetic Retinopathy Using Pretrained Convolutional Neural Networks , 2020, Complex..

[45]  A. Ganapathiraju,et al.  LINEAR DISCRIMINANT ANALYSIS - A BRIEF TUTORIAL , 1995 .

[46]  Pedro Costa,et al.  EyeWeS: Weakly Supervised Pre-Trained Convolutional Neural Networks for Diabetic Retinopathy Detection , 2019, 2019 16th International Conference on Machine Vision Applications (MVA).

[47]  김종영.  구글 TensorFlow 소개 , 2015 .

[48]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Jia Deng,et al.  A large-scale hierarchical image database , 2009, CVPR 2009.

[50]  Lin Li,et al.  A Deep Learning Method for Microaneurysm Detection in Fundus Images , 2016, 2016 IEEE First International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE).

[51]  A. Rakhlin.  Diabetic Retinopathy detection through integration of Deep Learning classification framework , 2017, bioRxiv.

[52]  Linda G. Shapiro,et al.  A SIFT descriptor with global context , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[53]  Waleed H. Abdulla,et al.  University of Auckland Diabetic Retinopathy (UoA-DR) Database- END USER LICENCE AGREEMENT , 2018 .

[54]  Muhammad Hussain,et al.  Deep Learning based Computer-Aided Diagnosis Systems for Diabetic Retinopathy: A Survey , 2018, Artif. Intell. Medicine.