Automated Quantification of Photoreceptor alteration in macular disease using Optical Coherence Tomography and Deep Learning

Diabetic macular edema (DME) and retina vein occlusion (RVO) are macular diseases in which central photoreceptors are affected due to pathological accumulation of fluid. Optical coherence tomography allows to visually assess and evaluate photoreceptor integrity, whose alteration has been observed as an important biomarker of both diseases. However, the manual quantification of this layered structure is challenging, tedious and time-consuming. In this paper we introduce a deep learning approach for automatically segmenting and characterising photoreceptor alteration. The photoreceptor layer is segmented using an ensemble of four different convolutional neural networks. En-face representations of the layer thickness are produced to characterize the photoreceptors. The pixel-wise standard deviation of the score maps produced by the individual models is also taken to indicate areas of photoreceptor abnormality or ambiguous results. Experimental results showed that our ensemble is able to produce results in pair with a human expert, outperforming each of its constitutive models. No statistically significant differences were observed between mean thickness estimates obtained from automated and manually generated annotations. Therefore, our model is able to reliable quantify photoreceptors, which can be used to improve prognosis and managment of macular diseases.

[1]  Xiaodong Wu,et al.  Automated 3-D Intraretinal Layer Segmentation of Macular Spectral-Domain Optical Coherence Tomography Images , 2009, IEEE Transactions on Medical Imaging.

[2]  Matthew B. Blaschko,et al.  A Discriminatively Trained Fully Connected Conditional Random Field Model for Blood Vessel Segmentation in Fundus Images , 2017, IEEE Transactions on Biomedical Engineering.

[3]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[4]  Qiang Chen,et al.  Beyond Retinal Layers: A Deep Voting Model for Automated Geographic Atrophy Segmentation in SD-OCT Images , 2018, Translational vision science & technology.

[5]  Raphael Sznitman,et al.  Pathological OCT Retinal Layer Segmentation Using Branch Residual U-Shape Networks , 2017, MICCAI.

[6]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[7]  W. Freeman,et al.  The association between percent disruption of the photoreceptor inner segment-outer segment junction and visual acuity in diabetic macular edema. , 2010, American journal of ophthalmology.

[8]  Jing Wu,et al.  A novel benchmark model for intelligent annotation of spectral-domain optical coherence tomography scans using the example of cyst annotation , 2016, Comput. Methods Programs Biomed..

[9]  Hyewon Chung,et al.  Association between photoreceptor integrity and visual outcome in diabetic macular edema , 2011, Graefe's Archive for Clinical and Experimental Ophthalmology.

[10]  Donald C Hood,et al.  A comparison of visual field sensitivity to photoreceptor thickness in retinitis pigmentosa. , 2010, Investigative ophthalmology & visual science.

[11]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[12]  Amir Sadeghipour,et al.  Artificial intelligence in retina , 2018, Progress in Retinal and Eye Research.

[13]  M. A. Hussein,et al.  OUTER RETINAL LAYER THICKNESS AS GOOD VISUAL PREDICTOR IN PATIENTS WITH DIABETIC MACULAR EDEMA , 2017, Retina.

[14]  Kazuaki Miyamoto,et al.  Foveal photoreceptor layer in eyes with persistent cystoid macular edema associated with branch retinal vein occlusion. , 2008, American journal of ophthalmology.

[15]  Andrew Hunter,et al.  The association between retinal vascular geometry changes and diabetic retinopathy and their role in prediction of progression – an exploratory study , 2014, BMC Ophthalmology.

[16]  I. Scott,et al.  Association of outer retinal layer morphology with visual acuity in patients with retinal vein occlusion: SCORE Study Report 13 , 2012, Eye.

[17]  Hariharan Ravishankar,et al.  Understanding the Mechanisms of Deep Transfer Learning for Medical Images , 2016, LABELS/DLMIA@MICCAI.

[18]  S. Dwivedi,et al.  Obesity May Be Bad: Compressed Convolutional Networks for Biomedical Image Segmentation , 2020 .

[19]  Hiroshi Tamura,et al.  Photoreceptor Damage and Reduction of Retinal Sensitivity Surrounding Geographic Atrophy in Age-Related Macular Degeneration. , 2016, American journal of ophthalmology.

[20]  Bianca S. Gerendas,et al.  Joint retinal layer and fluid segmentation in OCT scans of eyes with severe macular edema using unsupervised representation and auto-context. , 2017, Biomedical optics express.

[21]  M. Falcão,et al.  Outer retinal layers as predictors of visual acuity in retinitis pigmentosa: a cross-sectional study , 2018, Graefe's Archive for Clinical and Experimental Ophthalmology.

[22]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[23]  Geraint Rees,et al.  Clinically applicable deep learning for diagnosis and referral in retinal disease , 2018, Nature Medicine.

[24]  S. Sadda,et al.  Proposed lexicon for anatomic landmarks in normal posterior segment spectral-domain optical coherence tomography: the IN•OCT consensus. , 2014, Ophthalmology.

[25]  Tien Yin Wong,et al.  Diabetic macular oedema. , 2017, The lancet. Diabetes & endocrinology.

[26]  Pedagógia,et al.  Cross Sectional Study , 2019 .

[27]  Kenji Inoue,et al.  The association between photoreceptor layer thickness measured by optical coherence tomography and visual sensitivity in glaucomatous eyes , 2017, PloS one.

[28]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[29]  Bianca S. Gerendas,et al.  Fully Automated Detection and Quantification of Macular Fluid in OCT Using Deep Learning. , 2017, Ophthalmology.

[30]  Maria Wimmer,et al.  Fully Convolutional Architectures for Multiclass Segmentation in Chest Radiographs , 2017, IEEE Transactions on Medical Imaging.

[31]  Amir Sadeghipour,et al.  OCT biomarkers predictive for visual acuity in patients with diabetic macular edema , 2017 .

[32]  José Ignacio Orlando,et al.  U2-Net: A Bayesian U-Net Model With Epistemic Uncertainty Feedback For Photoreceptor Layer Segmentation In Pathological OCT Scans , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[33]  Konstantinos Kamnitsas,et al.  Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmentation , 2017, BrainLes@MICCAI.

[34]  Kentaro Yamamoto,et al.  RELATIONSHIP BETWEEN ABNORMALITIES OF PHOTORECEPTOR MICROSTRUCTURES AND MICROVASCULAR STRUCTURES DETERMINED BY OPTICAL COHERENCE TOMOGRAPHY ANGIOGRAPHY IN EYES WITH BRANCH RETINAL VEIN OCCLUSION. , 2018, Retina.

[35]  W. Freeman,et al.  INTEGRITY OF OUTER RETINAL LAYERS AFTER RESOLUTION OF CENTRAL INVOLVED DIABETIC MACULAR EDEMA , 2016, Retina.

[36]  U. Schmidt-Erfurth,et al.  A paradigm shift in imaging biomarkers in neovascular age-related macular degeneration , 2016, Progress in Retinal and Eye Research.

[37]  H. Ishikawa,et al.  QUANTIFICATION OF PHOTORECEPTOR LAYER THICKNESS IN NORMAL EYES USING OPTICAL COHERENCE TOMOGRAPHY , 2006, Retina.

[38]  Masahiro Fujimoto,et al.  Restoration of foveal photoreceptors after intravitreal ranibizumab injections for diabetic macular edema , 2016, Scientific Reports.