CANet: Cross-Disease Attention Network for Joint Diabetic Retinopathy and Diabetic Macular Edema Grading

Diabetic retinopathy (DR) and diabetic macular edema (DME) are the leading causes of permanent blindness in the working-age population. Automatic grading of DR and DME helps ophthalmologists design tailored treatments to patients, thus is of vital importance in the clinical practice. However, prior works either grade DR or DME, and ignore the correlation between DR and its complication, i.e., DME. Moreover, the location information, e.g., macula and soft hard exhaust annotations, are widely used as a prior for grading. Such annotations are costly to obtain, hence it is desirable to develop automatic grading methods with only image-level supervision. In this article, we present a novel cross-disease attention network (CANet) to jointly grade DR and DME by exploring the internal relationship between the diseases with only image-level supervision. Our key contributions include the disease-specific attention module to selectively learn useful features for individual diseases, and the disease-dependent attention module to further capture the internal relationship between the two diseases. We integrate these two attention modules in a deep network to produce disease-specific and disease-dependent features, and to maximize the overall performance jointly for grading DR and DME. We evaluate our network on two public benchmark datasets, i.e., ISBI 2018 IDRiD challenge dataset and Messidor dataset. Our method achieves the best result on the ISBI 2018 IDRiD challenge dataset and outperforms other methods on the Messidor dataset. Our code is publicly available at https://github.com/xmengli999/CANet.

[1]  Misgina Tsighe Hagos,et al.  Transfer Learning based Detection of Diabetic Retinopathy from Small Dataset , 2019, ArXiv.

[2]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[4]  Lawrence Carin,et al.  Observations and Lessons Learned From the Artificial Intelligence Studies for Diabetic Retinopathy Screening. , 2019, JAMA ophthalmology.

[5]  Yifan Peng,et al.  A multi-task deep learning model for the classification of Age-related Macular Degeneration , 2018, AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science.

[6]  Qiang Wu,et al.  Retinal Microaneurysm Detection Using Clinical Report Guided Multi-sieving CNN , 2017, MICCAI.

[7]  Kimmo Kaski,et al.  Deep Learning Fundus Image Analysis for Diabetic Retinopathy and Macular Edema Grading , 2019, Scientific Reports.

[8]  Chen-Yi Lee,et al.  Diabetic Retinopathy Detection Based on Deep Convolutional Neural Networks , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Rishab Gargeya,et al.  Automated Identification of Diabetic Retinopathy Using Deep Learning. , 2017, Ophthalmology.

[11]  Bian Wu,et al.  A Framework for Identifying Diabetic Retinopathy Based on Anti-noise Detection and Attention-Based Fusion , 2018, MICCAI.

[12]  Frank Hutter,et al.  SGDR: Stochastic Gradient Descent with Warm Restarts , 2016, ICLR.

[13]  Osmar R. Zaïane,et al.  Efficient multi-kernel multi-instance learning using weakly supervised and imbalanced data for diabetic retinopathy diagnosis , 2018, Comput. Medical Imaging Graph..

[14]  Xiaogang Wang,et al.  Zoom-in-Net: Deep Mining Lesions for Diabetic Retinopathy Detection , 2017, MICCAI.

[15]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[16]  Majid A. Al-Taee,et al.  Diabetic Macular Edema Grading Based on Deep Neural Networks , 2016 .

[17]  Bálint Antal,et al.  An ensemble-based system for automatic screening of diabetic retinopathy , 2014, Knowl. Based Syst..

[18]  Farida Cheriet,et al.  Red Lesion Detection Using Dynamic Shape Features for Diabetic Retinopathy Screening , 2016, IEEE Transactions on Medical Imaging.

[19]  Xiaogang Li,et al.  Convolutional neural networks based transfer learning for diabetic retinopathy fundus image classification , 2017, 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[20]  Abhishek Verma,et al.  New Deep Neural Nets for Fine-Grained Diabetic Retinopathy Recognition on Hybrid Color Space , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[21]  Shehzad Khalid,et al.  Detection and classification of retinal lesions for grading of diabetic retinopathy , 2014, Comput. Biol. Medicine.

[22]  Fabrice Mériaudeau,et al.  Indian Diabetic Retinopathy Image Dataset (IDRiD): A Database for Diabetic Retinopathy Screening Research , 2018, Data.

[23]  Wufeng Xue,et al.  Full left ventricle quantification via deep multitask relationships learning , 2018, Medical Image Anal..

[24]  Max A. Viergever,et al.  Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities , 2016, MICCAI.

[25]  Sheikh Muhammad Saiful Islam,et al.  Deep Learning based Early Detection and Grading of Diabetic Retinopathy Using Retinal Fundus Images , 2018, ArXiv.

[26]  Wei Zhao,et al.  Patient-specific reconstruction of volumetric computed tomography images from a single projection view via deep learning , 2019, Nature Biomedical Engineering.

[27]  Alex Varghese,et al.  Ensemble of Convolutional Neural Networks for Automatic Grading of Diabetic Retinopathy and Macular Edema , 2018, ArXiv.

[28]  G. Bresnick,et al.  DIABETIC MACULAR EDEMA: A REVIEW , 1986 .

[29]  C. M. Lim,et al.  Computer-based detection of diabetes retinopathy stages using digital fundus images , 2009, Proceedings of the Institution of Mechanical Engineers. Part H, Journal of engineering in medicine.

[30]  Lei Xing,et al.  Incorporating imaging information from deep neural network layers into image guided radiation therapy (IGRT). , 2019, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[31]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Bram van Ginneken,et al.  Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images , 2016, IEEE Transactions on Medical Imaging.

[33]  Lakshminarayanan Subramanian,et al.  Case for Automated Detection of Diabetic Retinopathy , 2010, AAAI Spring Symposium: Artificial Intelligence for Development.

[34]  Muhammad Younus Javed,et al.  Automated detection of exudates and macula for grading of diabetic macular edema , 2014, Comput. Methods Programs Biomed..

[35]  Matthew B. Blaschko,et al.  An ensemble deep learning based approach for red lesion detection in fundus images , 2017, Comput. Methods Programs Biomed..

[36]  Nitin Kumar,et al.  Kernel Generalized-Gaussian Mixture Model for Robust Abnormality Detection , 2017, MICCAI.

[37]  Keshab K. Parhi,et al.  DREAM: Diabetic Retinopathy Analysis Using Machine Learning , 2014, IEEE Journal of Biomedical and Health Informatics.

[38]  Ling Shao,et al.  Collaborative Learning of Semi-Supervised Segmentation and Classification for Medical Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Hao Chen,et al.  MTMR-Net: Multi-task Deep Learning with Margin Ranking Loss for Lung Nodule Analysis , 2018, DLMIA/ML-CDS@MICCAI.

[40]  Shenghua Gao,et al.  Multi-Cell Multi-Task Convolutional Neural Networks for Diabetic Retinopathy Grading , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[41]  Arup Das,et al.  Diabetic Macular Edema: Pathophysiology and Novel Therapeutic Targets. , 2015, Ophthalmology.

[42]  Yogesan Kanagasingam,et al.  Major automatic diabetic retinopathy screening systems and related core algorithms: a review , 2018, Machine Vision and Applications.

[43]  Bunyarit Uyyanonvara,et al.  Automatic Exudate Detection from Non-dilated Diabetic Retinopathy Retinal Images Using Fuzzy C-means Clustering , 2009, Sensors.

[44]  Jonathan Krause,et al.  Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy , 2017, Ophthalmology.

[45]  F. Akay,et al.  Diabetic Macular Edema , 1969, Pakistan journal of medical sciences.

[46]  Laude,et al.  FEEDBACK ON A PUBLICLY DISTRIBUTED IMAGE DATABASE: THE MESSIDOR DATABASE , 2014 .

[47]  Frans Coenen,et al.  Convolutional Neural Networks for Diabetic Retinopathy , 2016, MIUA.

[48]  Jacques Wainer,et al.  Beyond Lesion-Based Diabetic Retinopathy: A Direct Approach for Referral , 2017, IEEE Journal of Biomedical and Health Informatics.

[49]  Yann LeCun,et al.  Energy-based Generative Adversarial Network , 2016, ICLR.

[50]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[51]  M. Fraz,et al.  Computational Methods for Exudates Detection and Macular Edema Estimation in Retinal Images: A Survey , 2018, Archives of Computational Methods in Engineering.

[52]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[53]  Nishanthan Ramachandran,et al.  Diabetic retinopathy screening using deep neural network , 2018, Clinical & experimental ophthalmology.

[54]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Meindert Niemeijer,et al.  Evaluation of a computer-aided diagnosis system for diabetic retinopathy screening on public data. , 2011, Investigative ophthalmology & visual science.

[56]  J. Shaw,et al.  IDF Diabetes Atlas: Global estimates of diabetes prevalence for 2017 and projections for 2045. , 2018, Diabetes research and clinical practice.

[57]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[58]  Tao Li,et al.  Lesion Detection and Grading of Diabetic Retinopathy via Two-Stages Deep Convolutional Neural Networks , 2017, MICCAI.

[59]  Lei Xing,et al.  Fully automatic segmentation of type B aortic dissection from CTA images enabled by deep learning. , 2019, European journal of radiology.

[60]  U. Rajendra Acharya,et al.  Automated diabetic macular edema (DME) grading system using DWT, DCT Features and maculopathy index , 2017, Comput. Biol. Medicine.

[61]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[62]  Dazhe Zhao,et al.  Diabetic macular edema grading in retinal images using vector quantization and semi-supervised learning , 2018, Technology and health care : official journal of the European Society for Engineering and Medicine.

[63]  Shehzad Khalid,et al.  Fundus Images-Based Detection and Grading of Macular Edema Using Robust Macula Localization , 2018, IEEE Access.

[64]  Dimitris N. Metaxas,et al.  Deep multi-task and task-specific feature learning network for robust shape preserved organ segmentation , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).