DcardNet: Diabetic Retinopathy Classification at Multiple Levels Based on Structural and Angiographic Optical Coherence Tomography

Objective: Optical coherence tomography (OCT) and its angiography (OCTA) have several advantages for the early detection and diagnosis of diabetic retinopathy (DR). However, automated, complete DR classification frameworks based on both OCT and OCTA data have not been proposed. In this study, a convolutional neural network (CNN) based method is proposed to fulfill a DR classification framework using en face OCT and OCTA. Methods: A densely and continuously connected neural network with adaptive rate dropout (DcardNet) is designed for the DR classification. In addition, adaptive label smoothing was proposed and used to suppress overfitting. Three separate classification levels are generated for each case based on the International Clinical Diabetic Retinopathy scale. At the highest level the network classifies scans as referable or non-referable for DR. The second level classifies the eye as non-DR, non-proliferative DR (NPDR), or proliferative DR (PDR). The last level classifies the case as no DR, mild and moderate NPDR, severe NPDR, and PDR. Results: We used 10-fold cross-validation with 10% of the data to assess the network's performance. The overall classification accuracies of the three levels were 95.7%, 85.0%, and 71.0% respectively. Conclusion/Significance: A reliable, sensitive and specific automated classification framework for referral to an ophthalmologist can be a key technology for reducing vision loss related to DR.

[1]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[2]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[4]  David Huang,et al.  Reflectance-based projection-resolved optical coherence tomography angiography [Invited]. , 2017, Biomedical optics express.

[5]  David Huang,et al.  Automated Quantification of Capillary Nonperfusion Using Optical Coherence Tomography Angiography in Diabetic Retinopathy. , 2016, JAMA ophthalmology.

[6]  Frank Hutter,et al.  SGDR: Stochastic Gradient Descent with Warm Restarts , 2016, ICLR.

[7]  Yali Jia,et al.  Automated segmentation of peripapillary retinal boundaries in OCT combining a convolutional neural network and a multi-weights graph search. , 2019, Biomedical optics express.

[8]  David Huang,et al.  Advanced image processing for optical coherence tomographic angiography of macular diseases. , 2015, Biomedical optics express.

[9]  Sina Farsiu,et al.  Automatic segmentation of up to ten layer boundaries in SD-OCT images of the mouse retina with and without missing layers due to pathology. , 2014, Biomedical optics express.

[10]  David Huang,et al.  Projection-resolved optical coherence tomographic angiography. , 2016, Biomedical optics express.

[11]  Matthew D. Davis,et al.  Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales. , 2003, Ophthalmology.

[12]  Ayman El-Baz,et al.  Automated Diagnosis and Grading of Diabetic Retinopathy Using Optical Coherence Tomography , 2018, Investigative ophthalmology & visual science.

[13]  Fundus photographic risk factors for progression of diabetic retinopathy. ETDRS report number 12. Early Treatment Diabetic Retinopathy Study Research Group. , 1991, Ophthalmology.

[14]  Ayman El-Baz,et al.  Automated diabetic retinopathy detection using optical coherence tomography angiography: a pilot study , 2018, British Journal of Ophthalmology.

[15]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[16]  Xincheng Yao,et al.  QUANTITATIVE OPTICAL COHERENCE TOMOGRAPHY ANGIOGRAPHY FEATURES FOR OBJECTIVE CLASSIFICATION AND STAGING OF DIABETIC RETINOPATHY. , 2018, Retina.

[17]  Martin F. Kraus,et al.  Split-spectrum amplitude-decorrelation angiography with optical coherence tomography , 2012, Optics express.

[18]  Francesco Bandello,et al.  In vivo rotational three-dimensional OCTA analysis of microaneurysms in the human diabetic retina , 2019, Scientific Reports.

[19]  David Alonso-Caneiro,et al.  Automatic segmentation of OCT retinal boundaries using recurrent neural networks and graph search. , 2018, Biomedical optics express.

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Yue Wu,et al.  Deep-Learning Based, Automated Segmentation of Macular Edema in Optical Coherence Tomography , 2017, bioRxiv.

[22]  David J. Wilson,et al.  Detailed Vascular Anatomy of the Human Retina by Projection-Resolved Optical Coherence Tomography Angiography , 2017, Scientific Reports.

[23]  Amani A. Fawzi,et al.  Importance of Considering the Middle Capillary Plexus on OCT Angiography in Diabetic Retinopathy , 2018, Investigative ophthalmology & visual science.

[24]  G. Ripandelli,et al.  Optical coherence tomography. , 1998, Seminars in ophthalmology.

[25]  Yalin Zheng,et al.  Automated layer segmentation of macular OCT images via graph-based SLIC superpixels and manifold ranking approach , 2017, Comput. Medical Imaging Graph..

[26]  Sina Farsiu,et al.  Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema. , 2015, Biomedical optics express.

[27]  Jie Wang,et al.  Maximum value projection produces better en face OCT angiograms than mean value projection. , 2018, Biomedical optics express.

[28]  Kuntal Ghosh,et al.  Automatic detection and classification of diabetic retinopathy stages using CNN , 2017, 2017 4th International Conference on Signal Processing and Integrated Networks (SPIN).

[29]  Rishab Gargeya,et al.  Automated Identification of Diabetic Retinopathy Using Deep Learning. , 2017, Ophthalmology.

[30]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[31]  Isabelle Bloch,et al.  Automated segmentation of macular layers in OCT images and quantitative evaluation of performances , 2011, Pattern Recognit..

[32]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  M. Abràmoff,et al.  Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep Learning. , 2016, Investigative ophthalmology & visual science.

[34]  Geoffrey E. Hinton,et al.  Regularizing Neural Networks by Penalizing Confident Output Distributions , 2017, ICLR.

[35]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[36]  James G. Fujimoto,et al.  Quantitative 3D-OCT motion correction with tilt and illumination correction, robust similarity measure and regularization , 2014, Biomedical optics express.

[37]  Neil Genzlinger.  A. and Q , 2006 .

[38]  David J. Wilson,et al.  Quantitative optical coherence tomography angiography of vascular abnormalities in the living human eye , 2015, Proceedings of the National Academy of Sciences.

[39]  Gangjun Liu,et al.  Optimization of the split-spectrum amplitude-decorrelation angiography algorithm on a spectral optical coherence tomography system. , 2015, Optics letters.

[40]  Mirza Faisal Beg,et al.  Ensemble Deep Learning for Diabetic Retinopathy Detection Using Optical Coherence Tomography Angiography , 2020, Translational vision science & technology.

[41]  S. Haneda,et al.  [International clinical diabetic retinopathy disease severity scale]. , 2010, Nihon rinsho. Japanese journal of clinical medicine.

[42]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[43]  Jie Wang,et al.  Automated segmentation of retinal layer boundaries and capillary plexuses in wide-field optical coherence tomographic angiography , 2018, Biomedical optics express.

[44]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[45]  David J. Wilson,et al.  OPTICAL COHERENCE TOMOGRAPHY ANGIOGRAPHY FEATURES OF DIABETIC RETINOPATHY , 2015, Retina.

[46]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[47]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[48]  David Huang,et al.  Visualization of 3 Distinct Retinal Plexuses by Projection-Resolved Optical Coherence Tomography Angiography in Diabetic Retinopathy. , 2016, JAMA ophthalmology.

[49]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Thomas S. Hwang,et al.  Automated Quantification of Nonperfusion in Three Retinal Plexuses Using Projection-Resolved Optical Coherence Tomography Angiography in Diabetic Retinopathy , 2016, Investigative ophthalmology & visual science.