Joint Segmentation and Quantification of Chorioretinal Biomarkers in Optical Coherence Tomography Scans: A Deep Learning Approach

In ophthalmology, chorioretinal biomarkers (CRBMs) play a significant role in detecting, quantifying, and ameliorating the treatment of chronic eye conditions. Optical coherence tomography (OCT) imaging is primarily used for investigating various CRBMs and prompt intervention of retinal conditions. However, with extensive clinical applications and increasing prevalence of ocular diseases, the number of OCT scans obtained globally exceeds ophthalmologists’ capacity to examine these in a meaningful manner. Instead, the emergence of deep learning provides a cost-effective and reliable alternative for automated analysis of scans, assisting ophthalmologists in clinical routines and research. This article presents a residual learning-based framework (RASP-Net) that integrates atrous spatial pyramid pooling, coherent preprocessing, and postprocessing mechanisms to achieve joint segmentation and quantification of 11 CRBMs. We used a total of 7000 annotated scans for training, validation, and testing purposes of RASP-Net. Moreover, a novel algorithm for 3-D macular profiles reconstruction is presented to give a more intuitive way for characterizing the CRBMs based on coarse contouring and quantification. The proposed framework is evaluated through several experiments using different performance metrics. The results presented in this study validate the optimal performance of RASP-Net in precise detection and segmentation of CRBMs, with mean balanced accuracy, intersection over union, and dice score values of 0.916, 0.634, and 0.776 respectively. The proposed RASP-Net model characterizes a wide range of CRBMs with fine-grained pixelwise segmentation, extraction, and quantification in the context of retinal pathologies. This proposed system can allow retina experts to monitor the improvement and deterioration of the underlying ocular conditions.

[1]  Eric L Yuan,et al.  Quantitative classification of eyes with and without intermediate age-related macular degeneration using optical coherence tomography. , 2014, Ophthalmology.

[2]  T. Sano,et al.  [Diabetic retinopathy]. , 2001, Nihon rinsho. Japanese journal of clinical medicine.

[3]  Taimur Hassan,et al.  Structure tensor based automated detection of macular edema and central serous retinopathy using optical coherence tomography images. , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[4]  Alireza Mehridehnavi,et al.  Macular OCT Classification Using a Multi-Scale Convolutional Neural Network Ensemble , 2018, IEEE Transactions on Medical Imaging.

[5]  Bjoern H Menze,et al.  Deep neural network for automatic characterization of lesions on 68Ga-PSMA-11 PET/CT , 2019, European Journal of Nuclear Medicine and Molecular Imaging.

[6]  Konstantina S. Nikita,et al.  A modular software system to assist interpretation of medical images application to vascular ultrasound images , 2006, 2004 IEEE International Workshop on Imaging Systems and Techniques (IST) (IEEE Cat. No.04EX896).

[7]  Sijie Niu,et al.  RMPPNet: residual multiple pyramid pooling network for subretinal fluid segmentation in SD-OCT images , 2020, OSA Continuum.

[8]  Edmond S. L. Ho,et al.  A Multitier Deep Learning Model for Arrhythmia Detection , 2020, IEEE Transactions on Instrumentation and Measurement.

[9]  Shenghua Gao,et al.  CE-Net: Context Encoder Network for 2D Medical Image Segmentation , 2019, IEEE Transactions on Medical Imaging.

[10]  Chong Wang,et al.  Attention to Lesion: Lesion-Aware Convolutional Neural Network for Retinal Optical Coherence Tomography Image Classification , 2019, IEEE Transactions on Medical Imaging.

[11]  Yifan Jian,et al.  Real-time retinal layer segmentation of OCT volumes with GPU accelerated inferencing using a compressed, low-latency neural network. , 2020, Biomedical optics express.

[12]  Raphael Sznitman,et al.  Fused Detection of Retinal Biomarkers in OCT Volumes , 2019, MICCAI.

[13]  Keshab K. Parhi,et al.  RETOUCH: The Retinal OCT Fluid Detection and Segmentation Benchmark and Challenge , 2019, IEEE Transactions on Medical Imaging.

[14]  Bo Li,et al.  Computer Aided Diagnosis of Idiopathic Central Serous Chorioretinopathy , 2018, 2018 2nd IEEE Advanced Information Management,Communicates,Electronic and Automation Control Conference (IMCEC).

[15]  Sina Farsiu,et al.  Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images. , 2014, Biomedical optics express.

[16]  Hossein Rabbani,et al.  Analysis of a Novel Segmentation Algorithm for Optical Coherence Tomography Images Based on Pixels Intensity Correlations , 2021, IEEE Transactions on Instrumentation and Measurement.

[17]  Yonghong He,et al.  Synchronous Fingerprint Acquisition System Based on Total Internal Reflection and Optical Coherence Tomography , 2020, IEEE Transactions on Instrumentation and Measurement.

[18]  Qingge Ji,et al.  MDAN-UNet: Multi-Scale and Dual Attention Enhanced Nested U-Net Architecture for Segmentation of Optical Coherence Tomography Images , 2020, Algorithms.

[19]  Leyuan Fang,et al.  Unsupervised Denoising of Optical Coherence Tomography Images With Nonlocal-Generative Adversarial Network , 2021, IEEE Transactions on Instrumentation and Measurement.

[20]  Sang Min Yoon,et al.  Integrated Near-Infrared Dual-Modality Imaging With Common Path Fiber Optic System , 2011, IEEE Transactions on Instrumentation and Measurement.

[21]  Ying Zhang,et al.  Confidence-Guided Topology-Preserving Layer Segmentation for Optical Coherence Tomography Images With Focus-Column Module , 2021, IEEE Transactions on Instrumentation and Measurement.

[22]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Juan Xu,et al.  Three dimensional optical coherence tomography imaging: Advantages and advances , 2010, Progress in Retinal and Eye Research.

[24]  Youxin Mao,et al.  High-Quality Tissue Imaging Using a Catheter-Based Swept-Source Optical Coherence Tomography Systems With an Integrated Semiconductor Optical Amplifier , 2011, IEEE Transactions on Instrumentation and Measurement.

[25]  Karel J. Zuiderveld,et al.  Contrast Limited Adaptive Histogram Equalization , 1994, Graphics Gems.

[26]  Shutao Li,et al.  Segmentation Based Sparse Reconstruction of Optical Coherence Tomography Images , 2017, IEEE Transactions on Medical Imaging.

[27]  Rob Fergus,et al.  Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[28]  Taimur Hassan,et al.  Deep structure tensor graph search framework for automated extraction and characterization of retinal layers and fluid pathology in retinal SD-OCT scans , 2019, Comput. Biol. Medicine.

[29]  Vasudevan Lakshminarayanan,et al.  OCTID: Optical Coherence Tomography Image Database , 2018, Comput. Electr. Eng..

[30]  Yingjie Yu,et al.  Full-Range Line-Field Optical Coherence Tomography for High-Accuracy Measurements of Optical Lens , 2020, IEEE Transactions on Instrumentation and Measurement.

[31]  Taimur Hassan,et al.  Automated diagnosis of macular edema and central serous retinopathy through robust reconstruction of 3D retinal surfaces , 2016, Comput. Methods Programs Biomed..

[32]  Qiang Chen,et al.  MS-CAM: Multi-Scale Class Activation Maps for Weakly-Supervised Segmentation of Geographic Atrophy Lesions in SD-OCT Images , 2020, IEEE Journal of Biomedical and Health Informatics.

[33]  T. Lai,et al.  The potential of spectral domain optical coherence tomography imaging based retinal biomarkers , 2017, International Journal of Retina and Vitreous.

[34]  Hrvoje Bogunovic,et al.  U-Net with Spatial Pyramid Pooling for Drusen Segmentation in Optical Coherence Tomography , 2019, OMIA@MICCAI.

[35]  Qiang Chen,et al.  Fast and Automated Hyperreflective Foci Segmentation Based on Image Enhancement and Improved 3D U-Net in SD-OCT Volumes with Diabetic Retinopathy , 2020, Translational vision science & technology.

[36]  Adeel M. Syed,et al.  Automated segmentation of subretinal layers for the detection of macular edema. , 2016, Applied optics.

[37]  Xinjian Chen,et al.  Attention multi-scale network for pigment epithelial detachment segmentation in OCT images , 2020, Medical Imaging: Image Processing.

[38]  W. Sheu,et al.  Epiretinal Membrane Detection at the Ophthalmologist Level using Deep Learning of Optical Coherence Tomography , 2020, Scientific Reports.

[39]  Bilal Hassan,et al.  A comprehensive study capturing vision loss burden in Pakistan (1990-2025): Findings from the Global Burden of Disease (GBD) 2017 study , 2019, PloS one.

[40]  Sina Farsiu,et al.  Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema. , 2015, Biomedical optics express.

[41]  Bo Li,et al.  Deep Ensemble Learning Based Objective Grading of Macular Edema by Extracting Clinically Significant Findings from Fused Retinal Imaging Modalities , 2019, Sensors.

[42]  Gongping Yang,et al.  Automated segmentation of choroidal neovascularization in optical coherence tomography images using multi-scale convolutional neural networks with structure prior , 2017, Multimedia Systems.

[43]  Jost B Jonas,et al.  Incidence and progression of diabetic retinopathy: a systematic review. , 2019, The lancet. Diabetes & endocrinology.

[44]  Hossein Rabbani,et al.  Comparison of macular OCTs in right and left eyes of normal people , 2014, Medical Imaging.

[45]  Hon Keung Kwan,et al.  Fuzzy filters for image filtering , 2002, The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002..

[46]  Raphael Sznitman,et al.  Expert-level Automated Biomarker Identification in Optical Coherence Tomography Scans , 2019, Scientific Reports.

[47]  Shutao Li,et al.  Fast Acquisition and Reconstruction of Optical Coherence Tomography Images via Sparse Representation , 2013, IEEE Transactions on Medical Imaging.

[48]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Daniel S. Kermany,et al.  Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning , 2018, Cell.

[50]  Rangaraj M. Rangayyan,et al.  Parabolic Modeling of the Major Temporal Arcade in Retinal Fundus Images , 2012, IEEE Transactions on Instrumentation and Measurement.

[51]  Marek R. Ogiela,et al.  Nonlinear processing and semantic content analysis in medical imaging-a cognitive approach , 2005, IEEE Transactions on Instrumentation and Measurement.

[52]  Keshab K. Parhi,et al.  OCT Fluid Segmentation using Graph Shortest Path and Convolutional Neural Network* , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[53]  Jacob Scharcanski,et al.  Motion Detection and Compensation of Affine Deformations in Infrared Retinal Videos , 2017, IEEE Transactions on Instrumentation and Measurement.