Deep learning for predicting refractive error from retinal fundus images

Purpose We evaluate how deep learning can be applied to extract novel information such as refractive error from retinal fundus imaging. Methods Retinal fundus images used in this study were 45- and 30-degree field of view images from the UK Biobank and Age-Related Eye Disease Study (AREDS) clinical trials, respectively. Refractive error was measured by autorefraction in UK Biobank and subjective refraction in AREDS. We trained a deep learning algorithm to predict refractive error from a total of 226,870 images and validated it on 24,007 UK Biobank and 15,750 AREDS images. Our model used the "attention" method to identify features that are correlated with refractive error. Results The resulting algorithm had a mean absolute error (MAE) of 0.56 diopters (95% confidence interval [CI]: 0.55-0.56) for estimating spherical equivalent on the UK Biobank data set and 0.91 diopters (95% CI: 0.89-0.93) for the AREDS data set. The baseline expected MAE (obtained by simply predicting the mean of this population) was 1.81 diopters (95% CI: 1.79-1.84) for UK Biobank and 1.63 (95% CI: 1.60-1.67) for AREDS. Attention maps suggested that the foveal region was one of the most important areas used by the algorithm to make this prediction, though other regions also contribute to the prediction. Conclusions To our knowledge, the ability to estimate refractive error with high accuracy from retinal fundus photos has not been previously known and demonstrates that deep learning can be applied to make novel predictions from medical images.

[1]  R. Newcomb,et al.  Clinical investigation of the foveal light reflex. , 1981, American journal of optometry and physiological optics.

[2]  A. Sommer,et al.  Race-, age-, gender-, and refractive error-related differences in the normal optic disc. , 1994, Archives of ophthalmology.

[3]  A. Rudnicka,et al.  Magnification characteristics of fundus imaging systems. , 1998, Ophthalmology.

[4]  A. Hofman,et al.  Determinants of optic disc characteristics in a general population: The Rotterdam Study. , 1999, Ophthalmology.

[5]  Rich Caruana,et al.  Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping , 2000, NIPS.

[6]  L. Bour,et al.  Fundus photography for measurement of macular pigment density distribution in children. , 2002, Investigative ophthalmology & visual science.

[7]  P. Mitchell,et al.  Population prevalence of tilted optic disks and the relationship of this sign to refractive error. , 2002, American journal of ophthalmology.

[8]  D. Atchison,et al.  Shape of the retinal surface in emmetropia and myopia. , 2005, Investigative ophthalmology & visual science.

[9]  J. Jonas.  Optic disk size correlated with refractive error. , 2005, American journal of ophthalmology.

[10]  T. Mihashi,et al.  In Vivo Measurements of Cone Photoreceptor Spacing in Myopic Eyes from Images Obtained by an Adaptive Optics Fundus Camera , 2007, Japanese Journal of Ophthalmology.

[11]  P. Mitchell,et al.  Lens opacity and refractive influences on the measurement of retinal vascular fractal dimension , 2010, Acta ophthalmologica.

[12]  D. Pascolini,et al.  Global estimates of visual impairment: 2010 , 2011, British Journal of Ophthalmology.

[13]  A. James.  2010 , 2011, Philo of Alexandria: an Annotated Bibliography 2007-2016.

[14]  P. Mitchell,et al.  Influence of refractive error and axial length on retinal vessel geometric characteristics. , 2011, Investigative ophthalmology & visual science.

[15]  T. Wong,et al.  Influence of refractive error on optic disc topographic parameters: the singapore malay eye study. , 2011, American journal of ophthalmology.

[16]  K. Nakashima,et al.  [The Rotterdam study]. , 2011, Nihon rinsho. Japanese journal of clinical medicine.

[17]  K. Naidoo,et al.  Uncorrected refractive errors , 2012, Indian journal of ophthalmology.

[18]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[19]  Nigel M. Bolster,et al.  How the smartphone is driving the eye-health imaging revolution , 2014 .

[20]  P. Foster,et al.  Epidemiology of myopia , 2014, Eye.

[21]  A. Bastawrous,et al.  Development and Validation of a Smartphone-Based Visual Acuity Test (Peek Acuity) for Clinical Practice and Community-Based Fieldwork. , 2015, JAMA ophthalmology.

[22]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[23]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[24]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[25]  Lisa A. Ostrin,et al.  Refractive Error and Ocular Parameters: Comparison of Two SD-OCT Systems , 2015, Optometry and vision science : official publication of the American Academy of Optometry.

[26]  Maciej Czepita,et al.  Macular Pigment Optical Density and Ocular Pulse Amplitude in Subjects with Different Axial Lengths and Refractive Errors , 2015, Medical science monitor : international medical journal of experimental and clinical research.

[27]  Nigel M. Bolster,et al.  Clinical Validation of a Smartphone-Based Adapter for Optic Disc Imaging in Kenya. , 2016, JAMA ophthalmology.

[28]  Dayong Wang,et al.  Deep Learning for Identifying Metastatic Breast Cancer , 2016, ArXiv.

[29]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[31]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[32]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[33]  O. Stegle,et al.  Deep learning for computational biology , 2016, Molecular systems biology.

[34]  Tobias Elze,et al.  Age, ocular magnification, and circumpapillary retinal nerve fiber layer thickness , 2017, Journal of biomedical optics.

[35]  Neda Baniasadi,et al.  Associations between Optic Nerve Head–Related Anatomical Parameters and Refractive Error over the Full Range of Glaucoma Severity , 2017, Translational vision science & technology.

[36]  Tobias Elze,et al.  Ametropia, retinal anatomy, and OCT abnormality patterns in glaucoma. 2. Impacts of optic nerve head parameters , 2017, Journal of biomedical optics.

[37]  Rishab Gargeya,et al.  Automated Identification of Diabetic Retinopathy Using Deep Learning. , 2017, Ophthalmology.

[38]  Tobias Elze,et al.  Ametropia, retinal anatomy, and OCT abnormality patterns in glaucoma. 1. Impacts of refractive error and interartery angle , 2017, Journal of biomedical optics.

[39]  Aleksey Boyko,et al.  Detecting Cancer Metastases on Gigapixel Pathology Images , 2017, ArXiv.

[40]  David B Elliott,et al.  What is the appropriate gold standard test for refractive error? , 2017, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[41]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[42]  Gregory S. Corrado,et al.  Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning , 2017, ArXiv.

[43]  Michael V. McConnell,et al.  Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning , 2017, Nature Biomedical Engineering.

[44]  Neda Baniasadi,et al.  The Interrelationship between Refractive Error, Blood Vessel Anatomy, and Glaucomatous Visual Field Loss , 2018, Translational vision science & technology.