Beyond Performance Metrics: Automatic Deep Learning Retinal OCT Analysis Reproduces Clinical Trial Outcome.

PURPOSE To validate the efficacy of a fully automatic, deep learning-based segmentation algorithm beyond conventional performance metrics by measuring the primary outcome of a clinical trial for macular telangiectasia type 2 (MacTel2). DESIGN Evaluation of diagnostic test or technology. PARTICIPANTS A total of 92 eyes from 62 participants with MacTel2 from a phase 2 clinical trial (NCT01949324) randomized to 1 of 2 treatment groups METHODS: The ellipsoid zone (EZ) defect areas were measured on spectral domain OCT images of each eye at 2 time points (baseline and month 24) by a fully automatic, deep learning-based segmentation algorithm. The change in EZ defect area from baseline to month 24 was calculated and analyzed according to the clinical trial protocol. MAIN OUTCOME MEASURE Difference in the change in EZ defect area from baseline to month 24 between the 2 treatment groups. RESULTS The difference in the change in EZ defect area from baseline to month 24 between the 2 treatment groups measured by the fully automatic segmentation algorithm was 0.072±0.035 mm2 (P = 0.021). This was comparable to the outcome of the clinical trial using semiautomatic measurements by expert readers, 0.065±0.033 mm2 (P = 0.025). CONCLUSIONS The fully automatic segmentation algorithm was as accurate as semiautomatic expert segmentation to assess EZ defect areas and was able to reliably reproduce the statistically significant primary outcome measure of the clinical trial. This approach, to validate the performance of an automatic segmentation algorithm on the primary clinical trial end point, provides a robust gauge of its clinical applicability.

[1]  E. Chew,et al.  Ciliary neurotrophic factor for macular telangiectasia type 2: results from a phase 1 safety trial. , 2015, American journal of ophthalmology.

[2]  Thomas Theelen,et al.  Deep learning approach for the detection and quantification of intraretinal cystoid fluid in multivendor optical coherence tomography. , 2018, Biomedical optics express.

[3]  Justis P. Ehlers,et al.  Volumetric ellipsoid zone mapping for enhanced visualisation of outer retinal integrity with optical coherence tomography , 2015, British Journal of Ophthalmology.

[4]  M. He,et al.  Efficacy of a Deep Learning System for Detecting Glaucomatous Optic Neuropathy Based on Color Fundus Photographs. , 2018, Ophthalmology.

[5]  I. Constable,et al.  Effect of Ciliary Neurotrophic Factor on Retinal Neurodegeneration in Patients with Macular Telangiectasia Type 2: A Randomized Clinical Trial. , 2019, Ophthalmology.

[6]  Joseph A. Izatt,et al.  Automatic segmentation of seven retinal layers in SDOCT images congruent with expert manual segmentation , 2010, Optics express.

[7]  Bianca S. Gerendas,et al.  Machine Learning to Analyze the Prognostic Value of Current Imaging Biomarkers in Neovascular Age-Related Macular Degeneration. , 2018, Ophthalmology. Retina.

[8]  Eric J Topol,et al.  High-performance medicine: the convergence of human and artificial intelligence , 2019, Nature Medicine.

[9]  G. R. Jackson,et al.  Multimodal characterization of proliferative diabetic retinopathy reveals alterations in outer retinal function and structure. , 2015, Ophthalmology.

[10]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[11]  Sina Farsiu,et al.  Validated automatic segmentation of AMD pathology including drusen and geographic atrophy in SD-OCT images. , 2012, Investigative ophthalmology & visual science.

[12]  Qiang Chen,et al.  Beyond Retinal Layers: A Deep Voting Model for Automated Geographic Atrophy Segmentation in SD-OCT Images , 2018, Translational vision science & technology.

[13]  M. Mukaka,et al.  Statistics corner: A guide to appropriate use of correlation coefficient in medical research. , 2012, Malawi medical journal : the journal of Medical Association of Malawi.

[14]  M. Abràmoff,et al.  Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices , 2018, npj Digital Medicine.

[15]  Irene Leung,et al.  CORRELATION OF STRUCTURAL AND FUNCTIONAL OUTCOME MEASURES IN A PHASE ONE TRIAL OF CILIARY NEUROTROPHIC FACTOR IN TYPE 2 IDIOPATHIC MACULAR TELANGIECTASIA , 2017, Retina.

[16]  E. Finkelstein,et al.  Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes , 2017, JAMA.

[17]  Emily Y. Chew,et al.  Macular telangiectasia type 2 , 2013, Progress in Retinal and Eye Research.

[18]  Yue Wu,et al.  Deep-Learning Based, Automated Segmentation of Macular Edema in Optical Coherence Tomography , 2017, bioRxiv.

[19]  Xinjian Chen,et al.  Automatic Three-dimensional Detection of Photoreceptor Ellipsoid Zone Disruption Caused by Trauma in the OCT , 2016, Scientific reports.

[20]  Pierre Soille,et al.  Morphological Image Analysis: Principles and Applications , 2003 .

[21]  Yue Yu,et al.  Deep Learning-Based Automated Classification of Multi-Categorical Abnormalities From Optical Coherence Tomography Images , 2018, Translational vision science & technology.

[22]  Dengwang Li,et al.  Automated detection of photoreceptor disruption in mild diabetic retinopathy on volumetric optical coherence tomography. , 2017, Biomedical optics express.

[23]  Rishab Gargeya,et al.  Automated Identification of Diabetic Retinopathy Using Deep Learning. , 2017, Ophthalmology.

[24]  Bianca S. Gerendas,et al.  Fully Automated Detection and Quantification of Macular Fluid in OCT Using Deep Learning. , 2017, Ophthalmology.

[25]  Sina Farsiu,et al.  Macular sub-layer thinning and association with pulmonary function tests in Amyotrophic Lateral Sclerosis , 2016, Scientific Reports.

[26]  James M. Brown,et al.  Automated Fundus Image Quality Assessment in Retinopathy of Prematurity Using Deep Convolutional Neural Networks. , 2019, Ophthalmology. Retina.

[27]  Chong Wang,et al.  Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. , 2017, Biomedical optics express.

[28]  Luis de Sisternes,et al.  Visual Prognosis of Eyes Recovering From Macular Hole Surgery Through Automated Quantitative Analysis of Spectral-Domain Optical Coherence Tomography (SD-OCT) Scans. , 2015, Investigative ophthalmology & visual science.

[29]  Yifan Peng,et al.  DeepSeeNet: A deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs , 2018, Ophthalmology.

[30]  Sina Farsiu,et al.  Drusen Volume and Retinal Pigment Epithelium Abnormal Thinning Volume Predict 2-Year Progression of Age-Related Macular Degeneration. , 2016, Ophthalmology.

[31]  A. Peters,et al.  A Deep Learning Algorithm for Prediction of Age-Related Eye Disease Study Severity Scale for Age-Related Macular Degeneration from Color Fundus Photography. , 2018, Ophthalmology.

[32]  Marinko V Sarunic,et al.  Deep‐learning based multiclass retinal fluid segmentation and detection in optical coherence tomography images using a fully convolutional neural network , 2019, Medical Image Anal..

[33]  Sina Farsiu,et al.  Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema. , 2015, Biomedical optics express.

[34]  Neil J. Joshi,et al.  Automated Grading of Age-Related Macular Degeneration From Color Fundus Images Using Deep Convolutional Neural Networks , 2017, JAMA ophthalmology.

[35]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[36]  Irene Leung,et al.  CORRELATION OF CLINICAL AND STRUCTURAL PROGRESSION WITH VISUAL ACUITY LOSS IN MACULAR TELANGIECTASIA TYPE 2: MacTel Project Report No. 6–The MacTel Research Group , 2017, Retina.

[37]  Omer P. Kocaoglu,et al.  The cellular origins of the outer retinal bands in optical coherence tomography images. , 2014, Investigative ophthalmology & visual science.

[38]  Sina Farsiu,et al.  Longitudinal Associations Between Microstructural Changes and Microperimetry in the Early Stages of Age-Related Macular Degeneration. , 2016, Investigative ophthalmology & visual science.

[39]  M. Abràmoff,et al.  Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep Learning. , 2016, Investigative ophthalmology & visual science.

[40]  Jie Wang,et al.  Deep learning for the segmentation of preserved photoreceptors on en face optical coherence tomography in two inherited retinal diseases. , 2018, Biomedical optics express.

[41]  Justis P. Ehlers,et al.  Ellipsoid Zone Mapping Parameters In Retinal Venous Occlusive Disease With Associated Macular Edema. , 2018, Ophthalmology. Retina.

[42]  Thomas Theelen,et al.  Robust total retina thickness segmentation in optical coherence tomography images using convolutional neural networks. , 2017, Biomedical optics express.

[43]  Aaron Y. Lee,et al.  Deep learning is effective for the classification of OCT images of normal versus Age-related Macular Degeneration , 2016, bioRxiv.

[44]  Sina Farsiu,et al.  Deep longitudinal transfer learning-based automatic segmentation of photoreceptor ellipsoid zone defects on optical coherence tomography images of macular telangiectasia type 2 , 2018, Biomedical optics express.

[45]  Aaron Y. Lee,et al.  Deep learning is effective for the classification of OCT images of normal versus Age-related Macular Degeneration , 2016, bioRxiv.

[46]  Nassir Navab,et al.  ReLayNet: Retinal Layer and Fluid Segmentation of Macular Optical Coherence Tomography using Fully Convolutional Network , 2017, Biomedical optics express.

[47]  James M. Brown,et al.  Automated Diagnosis of Plus Disease in Retinopathy of Prematurity Using Deep Convolutional Neural Networks , 2018, JAMA ophthalmology.

[48]  Xiaodong Wu,et al.  Automated 3-D Intraretinal Layer Segmentation of Macular Spectral-Domain Optical Coherence Tomography Images , 2009, IEEE Transactions on Medical Imaging.

[49]  Xinjian Chen,et al.  Automated 3-D Retinal Layer Segmentation of Macular Optical Coherence Tomography Images With Serous Pigment Epithelial Detachments , 2015, IEEE Transactions on Medical Imaging.

[50]  Eric L Yuan,et al.  Quantitative classification of eyes with and without intermediate age-related macular degeneration using optical coherence tomography. , 2014, Ophthalmology.

[51]  Geraint Rees,et al.  Clinically applicable deep learning for diagnosis and referral in retinal disease , 2018, Nature Medicine.

[52]  Mahnaz Shahidi,et al.  Enface Thickness Mapping and Reflectance Imaging of Retinal Layers in Diabetic Retinopathy , 2015, PloS one.

[53]  Gábor Márk Somfai,et al.  Real-Time Automatic Segmentation of Optical Coherence Tomography Volume Data of the Macular Region , 2015, PloS one.

[54]  Sina Farsiu,et al.  Correlation Between Macular Integrity Assessment and Optical Coherence Tomography Imaging of Ellipsoid Zone in Macular Telangiectasia Type 2 , 2017, Investigative ophthalmology & visual science.