Hyper-reflective Foci Segmentation in SD-OCT Retinal Images with Diabetic Retinopathy using Deep Convolutional Neural Networks.

PURPOSE To automatically and accurately segment hyper-reflective foci (HRF) in spectral domain optical coherence tomography (SD-OCT) images with diabetic retinopathy (DR) using deep convolutional neural networks. METHODS An automatic HRF segmentation model for SD-OCT images based on deep networks was constructed. The model segmented small lesions through pixel-wise predictions based on small image patches. We used an approach for discriminative features extraction for small patches by introducing small kernels and strides in convolutional and pooling layers, which was applied on the state-of-the-art deep classification networks (GoogLeNet and ResNet). The features extracted by the adapted deep networks were fed into a softmax layer to produce the probabilities of HRF. We trained different models on a dataset with 16 HRF eyes by using different sizes of patches, and then, we fused these models to generate optimal results. RESULTS Experimental results on 18 eyes demonstrated that our method is effective for the HRF segmentation. The dice similarity coefficient (DSC) for the foci area in B-scan, projection images and foci amount in B-scan images reach 67.81%, 74.09%, and 72.45%, respectively. CONCLUSIONS The proposed segmentation model can accurately segment HRF in SD-OCT images with DR and outperforms traditional methods. Our model may provide reliable segmentations for small lesions in SD-OCT images and may be helpful in the clinical diagnosis of diseases. This article is protected by copyright. All rights reserved.

[1]  Sijie Niu,et al.  Multimodality analysis of Hyper-reflective Foci and Hard Exudates in Patients with Diabetic Retinopathy , 2017, Scientific Reports.

[2]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Nagahisa Yoshimura,et al.  Optical coherence tomographic evaluation of foveal hard exudates in patients with diabetic maculopathy accompanying macular detachment. , 2010, Ophthalmology.

[4]  Fucang Jia,et al.  Automatic Segmentation of Liver Tumor in CT Images with Deep Convolutional Neural Networks , 2015 .

[5]  R. Klein,et al.  The Wisconsin Epidemiologic Study of Diabetic Retinopathy. XV. The long-term incidence of macular edema. , 1995, Ophthalmology.

[6]  Nagahisa Yoshimura,et al.  Association between Hyperreflective Foci in the Outer Retina, Status of Photoreceptor Layer and Visual Acuity in Diabetic Macular Edema Running title: Hyperreflective foci, photoreceptor layer, and VA in DME , 2018 .

[7]  Qiang Chen,et al.  Application of Improved Homogeneity Similarity-Based Denoising in Optical Coherence Tomography Retinal Images , 2015, Journal of Digital Imaging.

[8]  Edoardo Midena,et al.  HYPERREFLECTIVE INTRARETINAL SPOTS IN RADIATION MACULAR EDEMA ON SPECTRAL DOMAIN OPTICAL COHERENCE TOMOGRAPHY , 2016, Retina.

[9]  Kazuaki Miyamoto,et al.  CHARACTERISTICS OF OPTICAL COHERENCE TOMOGRAPHIC HYPERREFLECTIVE FOCI IN RETINAL VEIN OCCLUSION , 2012, Retina.

[10]  Shuiwang Ji,et al.  Deep convolutional neural networks for multi-modality isointense infant brain image segmentation , 2015, NeuroImage.

[11]  Joseph A. Izatt,et al.  Automatic segmentation of seven retinal layers in SDOCT images congruent with expert manual segmentation , 2010, Optics express.

[12]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[13]  Zhi-Hua Zhou,et al.  Exploratory Under-Sampling for Class-Imbalance Learning , 2006, ICDM.

[14]  Luis de Sisternes,et al.  Fully Automated Prediction of Geographic Atrophy Growth Using Quantitative Spectral-Domain Optical Coherence Tomography Biomarkers. , 2016, Ophthalmology.

[15]  Matthias Bolz,et al.  Effect of retinal photocoagulation on intraretinal lipid exudates in diabetic macular edema documented by optical coherence tomography. , 2010, Ophthalmology.

[16]  D L DeMets,et al.  The Wisconsin epidemiologic study of diabetic retinopathy. XI. The incidence of macular edema. , 1989, Ophthalmology.

[17]  Burak Turgut,et al.  The Causes of Hyperreflective Dots in Optical Coherence Tomography Excluding Diabetic Macular Edema and Retinal Venous Occlusion§§ , 2015, The open ophthalmology journal.

[18]  Matthias Bolz,et al.  Optical coherence tomographic hyperreflective foci: a morphologic sign of lipid extravasation in diabetic macular edema. , 2009, Ophthalmology.

[19]  N. Congdon,et al.  Important causes of visual impairment in the world today. , 2003, JAMA.

[20]  Hyewon Chung,et al.  CORRELATION OF OPTICAL COHERENCE TOMOGRAPHIC HYPERREFLECTIVE FOCI WITH VISUAL OUTCOMES IN DIFFERENT PATTERNS OF DIABETIC MACULAR EDEMA , 2016, Retina.

[21]  B. Lujan,et al.  Progression of geographic atrophy in age-related macular degeneration imaged with spectral domain optical coherence tomography. , 2011, Ophthalmology.

[22]  Matthias Bolz,et al.  Detection and analysis of hard exudates by polarization-sensitive optical coherence tomography in patients with diabetic maculopathy. , 2014, Investigative ophthalmology & visual science.

[23]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[24]  Frans Coenen,et al.  Convolutional Neural Networks for Diabetic Retinopathy , 2016, MIUA.

[25]  Edoardo Midena,et al.  Retinal Layers Changes in Human Preclinical and Early Clinical Diabetic Retinopathy Support Early Retinal Neuronal and Müller Cells Alterations , 2013, Journal of diabetes research.

[26]  Max A. Viergever,et al.  Automatic Segmentation of MR Brain Images With a Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[27]  Makoto Nakamura,et al.  The ability of macular parameters and circumpapillary retinal nerve fiber layer by three SD-OCT instruments to diagnose highly myopic glaucoma. , 2013, Investigative ophthalmology & visual science.

[28]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[29]  T W Gardner,et al.  Diabetic retinopathy. , 1998, Diabetes care.

[30]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[31]  Xiangrong Zhou,et al.  Classification of teeth in cone-beam CT using deep convolutional neural network , 2017, Comput. Biol. Medicine.

[32]  Qiang Chen,et al.  Automated segmentation of hyperreflective foci in spectral domain optical coherence tomography with diabetic retinopathy , 2018, Journal of medical imaging.

[33]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Ronald M. Summers,et al.  Deep convolutional networks for pancreas segmentation in CT imaging , 2015, Medical Imaging.

[35]  J. Michael Jumper,et al.  SENSITIVITY OF FLUID DETECTION IN PATIENTS WITH NEOVASCULAR AMD USING SPECTRAL DOMAIN OPTICAL COHERENCE TOMOGRAPHY HIGH-DEFINITION LINE SCANS , 2014, Retina.

[36]  P. Zimmet,et al.  The worldwide epidemiology of type 2 diabetes mellitus—present and future perspectives , 2012, Nature Reviews Endocrinology.