Evaluation of Deep Segmentation Models for the Extraction of Retinal Lesions from Multi-modal Retinal Images

Identification of lesions plays a vital role in the accurate classification of retinal diseases and in helping clinicians analyzing the disease severity. In this paper, we present a detailed evaluation of RAGNet, PSPNet, SegNet, UNet, FCN-8 and FCN-32 for the extraction of retinal lesions such as intra-retinal fluid, sub-retinal fluid, hard exudates, drusen, and other chorioretinal anomalies from retinal fundus and OCT scans. We also discuss the transferability of these models for extracting retinal lesions by varying training-testing dataset pairs. A total of 363 fundus and 173,915 OCT scans were considered in this evaluation from seven publicly available datasets from which 297 fundus and 59,593 OCT scans were used for testing purposes. Overall, the best performance is achieved by RAGNet with a mean dice coefficient ($\mathrm{D_C}$) score of 0.822 for extracting retinal lesions. The second-best performance is achieved by PSPNet (mean $\mathrm{D_C}$: 0.785) using ResNet\textsubscript{50} as a backbone. Moreover, the best performance for extracting drusen is achieved by UNet ($\mathrm{D_C}$: 0.864). The source code is available at: this http URL.

[1]  Muhammad Younus Javed,et al.  An Automated System for the Grading of Diabetic Maculopathy in Fundus Images , 2012, ICONIP.

[2]  Taimur Hassan,et al.  Automated Segmentation and Quantification of Drusen in Fundus and Optical Coherence Tomography Images for Detection of ARMD , 2018, Journal of Digital Imaging.

[3]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Georg Langs,et al.  Exploiting Epistemic Uncertainty of Anatomy Segmentation for Anomaly Detection in Retinal OCT , 2019, IEEE Transactions on Medical Imaging.

[5]  Taimur Hassan,et al.  RAG-FW: A Hybrid Convolutional Framework for the Automated Extraction of Retinal Lesions and Lesion-Influenced Grading of Human Retinal Pathology , 2020, IEEE Journal of Biomedical and Health Informatics.

[6]  Bianca S. Gerendas,et al.  Fully Automated Detection and Quantification of Macular Fluid in OCT Using Deep Learning. , 2017, Ophthalmology.

[7]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Marcos Ortega,et al.  Intraretinal fluid identification via enhanced maps using optical coherence tomography images. , 2018, Biomedical optics express.

[9]  Daniel S. Kermany,et al.  Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning , 2018, Cell.

[10]  Taimur Hassan,et al.  BIOMISA Retinal Image Database for Macular and Ocular Syndromes , 2018, ICIAR.

[11]  Sina Farsiu,et al.  Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images. , 2014, Biomedical optics express.

[12]  Chong Wang,et al.  Attention to Lesion: Lesion-Aware Convolutional Neural Network for Retinal Optical Coherence Tomography Image Classification , 2019, IEEE Transactions on Medical Imaging.

[13]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[14]  Nidhi Relhan,et al.  The Early Treatment Diabetic Retinopathy Study historical review and relevance to today's management of diabetic macular edema , 2017, Current opinion in ophthalmology.

[15]  Alireza Mehridehnavi,et al.  Macular OCT Classification Using a Multi-Scale Convolutional Neural Network Ensemble , 2018, IEEE Transactions on Medical Imaging.

[16]  Nassir Navab,et al.  ReLayNet: Retinal Layer and Fluid Segmentation of Macular Optical Coherence Tomography using Fully Convolutional Network , 2017, Biomedical optics express.

[17]  Eric L Yuan,et al.  Quantitative classification of eyes with and without intermediate age-related macular degeneration using optical coherence tomography. , 2014, Ophthalmology.

[18]  Taimur Hassan,et al.  Automated diagnosis of macular edema and central serous retinopathy through robust reconstruction of 3D retinal surfaces , 2016, Comput. Methods Programs Biomed..

[19]  Taimur Hassan,et al.  Review of OCT and fundus images for detection of Macular Edema , 2015, 2015 IEEE International Conference on Imaging Systems and Techniques (IST).

[20]  Hossein Rabbani,et al.  Comparison of macular OCTs in right and left eyes of normal people , 2014, Medical Imaging.

[21]  Xinjian Chen,et al.  Automatic Retinal Layer Segmentation of OCT Images With Central Serous Retinopathy , 2019, IEEE Journal of Biomedical and Health Informatics.

[22]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  F. Akay,et al.  Diabetic Macular Edema , 1969, Pakistan journal of medical sciences.

[24]  Adeel M. Syed,et al.  Automated segmentation of subretinal layers for the detection of macular edema. , 2016, Applied optics.

[25]  Sina Farsiu,et al.  Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema. , 2015, Biomedical optics express.

[26]  Marilita M Moschos,et al.  Cystoid macular edema , 2008, Clinical Ophthalmology.

[27]  Chong Wang,et al.  Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. , 2017, Biomedical optics express.

[28]  Taimur Hassan,et al.  Structure tensor based automated detection of macular edema and central serous retinopathy using optical coherence tomography images. , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[29]  Taimur Hassan,et al.  Fully Automated Robust System to Detect Retinal Edema, Central Serous Chorioretinopathy, and Age Related Macular Degeneration from Optical Coherence Tomography Images , 2017, BioMed research international.

[30]  Amy L. Oldenburg,et al.  Automated Segmentation of Intraretinal Cystoid Fluid in Optical Coherence Tomography , 2012, IEEE Transactions on Biomedical Engineering.

[31]  Bo Li,et al.  Deep Ensemble Learning Based Objective Grading of Macular Edema by Extracting Clinically Significant Findings from Fused Retinal Imaging Modalities , 2019, Sensors.