Contrastive Learning Driven Self-Supervised Framework for Segmentation of Biomarker of Diabetic Macular Edema

Optical coherence tomography (OCT) has been widely used to investigate the pathological changes due to Diabetic Macular Edema (DME). In this paper, we developed a two-stage self-supervised learning approach to extract DME biomarkers. The backbone of the proposed scheme is the RAG-Net model trained in the first stage to extract different DME biomarkers, such as intra-retinal fluid, sub-retinal fluid, and hard exudates, using the low-shot supervision from the Zhang dataset. In the second training stage, it learns to extract the same biomarkers across the Duke-II dataset in a self-supervised manner (i.e., without using any ground-truth annotations) via the triplet loss function. We validated the proposed approach across both datasets at the inference stage, where it achieved the mean $\mathbf{IoU}$ score of 0.7610, and 0.7232, outperforming the state-of-the-art by 1.734%, and 4.839% for extracting DME biomarkers irrespective of the scanner specifications, and vendor artifacts across the Zhang and Duke-II datasets, respectively. In addition, we employed an external BIOMISA dataset to evaluate the proposed model and obtained an IOU of 0.752. The suggested model has the potential to improve clinical practice by allowing more objective assessment of DME patients. In the future, we plan to expand the application of our self-supervised model to other ocular diseases, such as glaucoma and AMD, and to incorporate more imaging modalities, such as fundus photographs and OCT-Angiography (OCTA).

[1]  Taimur Hassan,et al.  Continual Learning Objective for Analyzing Complex Knowledge Representations , 2022, Sensors.

[2]  Taimur Hassan,et al.  Incremental Cross-Domain Adaptation for Robust Retinopathy Screening via Bayesian Deep Learning , 2021, IEEE Transactions on Instrumentation and Measurement.

[3]  Taimur Hassan,et al.  Deep learning based joint segmentation and characterization of multi-class retinal fluid lesions on OCT scans for clinical use in anti-VEGF therapy , 2021, Comput. Biol. Medicine.

[4]  Taimur Hassan,et al.  RAG-FW: A Hybrid Convolutional Framework for the Automated Extraction of Retinal Lesions and Lesion-Influenced Grading of Human Retinal Pathology , 2020, IEEE Journal of Biomedical and Health Informatics.

[5]  Yonglong Tian,et al.  Contrastive Representation Distillation , 2019, ICLR.

[6]  Taimur Hassan,et al.  Fully automated detection, grading and 3D modeling of maculopathy from OCT volumes , 2019, 2019 2nd International Conference on Communication, Computing and Digital systems (C-CODE).

[7]  Taimur Hassan,et al.  Automated Segmentation and Quantification of Drusen in Fundus and Optical Coherence Tomography Images for Detection of ARMD , 2018, Journal of Digital Imaging.

[8]  Taimur Hassan,et al.  Fully Convolutional Neural Network for Lungs Segmentation from Chest X-Rays , 2018, ICIAR.

[9]  Taimur Hassan,et al.  BIOMISA Retinal Image Database for Macular and Ocular Syndromes , 2018, ICIAR.

[10]  Naoufel Werghi,et al.  Ground Moving Radar Targets Classification Based on Spectrogram Images Using Convolutional Neural Networks , 2018, 2018 19th International Radar Symposium (IRS).

[11]  Naoufel Werghi,et al.  Transfer learning with convolutional neural networks for moving target classification with micro-Doppler radar spectrograms , 2018, 2018 International Conference on Artificial Intelligence and Big Data (ICAIBD).

[12]  Daniel S. Kermany,et al.  Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning , 2018, Cell.

[13]  Bianca S. Gerendas,et al.  Fully Automated Detection and Quantification of Macular Fluid in OCT Using Deep Learning. , 2017, Ophthalmology.

[14]  Nassir Navab,et al.  ReLayNet: Retinal Layer and Fluid Segmentation of Macular Optical Coherence Tomography using Fully Convolutional Network , 2017, Biomedical optics express.

[15]  Debjani Chakraborty,et al.  Transfer learning based classification of optical coherence tomography images with diabetic macular edema and dry age-related macular degeneration. , 2017, Biomedical optics express.

[16]  Taimur Hassan,et al.  Fully automated diagnosis of papilledema through robust extraction of vascular patterns and ocular pathology from fundus photographs. , 2017, Biomedical optics express.

[17]  Gulistan Raja,et al.  Fully automated assessment of Macular Edema using Optical Coherence Tomography (OCT) images , 2016, 2016 International Conference on Intelligent Systems Engineering (ICISE).

[18]  Taimur Hassan,et al.  Structure tensor based automated detection of macular edema and central serous retinopathy using optical coherence tomography images. , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[19]  Alberto Del Bimbo,et al.  Local binary patterns on triangular meshes: Concept and applications , 2015, Comput. Vis. Image Underst..

[20]  Alberto Del Bimbo,et al.  Representing 3D texture on mesh manifolds for retrieval and recognition applications , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Xinjian Chen,et al.  Automated segmentation of intraretinal cystoid macular edema for retinal 3D OCT images with macular hole , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[22]  Sina Farsiu,et al.  Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema. , 2015, Biomedical optics express.

[23]  Shyam Vasudeva Rao,et al.  Segmentation and analysis of retinal layers (ILM & RPE) in Optical Coherence Tomography images with Edema , 2014, 2014 IEEE Conference on Biomedical Engineering and Sciences (IECBES).

[24]  Adisom Leelasantitham,et al.  Automated classification between age-related macular degeneration and Diabetic macular edema in OCT image using image segmentation , 2014, The 7th 2014 Biomedical Engineering International Conference.

[25]  Sina Farsiu,et al.  Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images. , 2014, Biomedical optics express.

[26]  Muhammad Younus Javed,et al.  Automated detection of exudates and macula for grading of diabetic macular edema , 2014, Comput. Methods Programs Biomed..

[27]  Amy L. Oldenburg,et al.  Automated Segmentation of Intraretinal Cystoid Fluid in Optical Coherence Tomography , 2012, IEEE Transactions on Biomedical Engineering.

[28]  Kai Hu,et al.  Embedded Residual Recurrent Network and Graph Search for the Segmentation of Retinal Layer Boundaries in Optical Coherence Tomography , 2021, IEEE Transactions on Instrumentation and Measurement.

[29]  Taimur Hassan,et al.  Joint Segmentation and Quantification of Chorioretinal Biomarkers in Optical Coherence Tomography Scans: A Deep Learning Approach , 2021, IEEE Transactions on Instrumentation and Measurement.

[30]  Naoufel Werghi,et al.  Emotion Recognition From Speech Using Wavelet Packet Transform Cochlear Filter Bank and Random Forest Classifier , 2020, IEEE Access.