Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study.

BACKGROUND Radical measures are required to identify and reduce blindness due to diabetes to achieve the Sustainable Development Goals by 2030. Therefore, we evaluated the accuracy of an artificial intelligence (AI) model using deep learning in a population-based diabetic retinopathy screening programme in Zambia, a lower-middle-income country. METHODS We adopted an ensemble AI model consisting of a combination of two convolutional neural networks (an adapted VGGNet architecture and a residual neural network architecture) for classifying retinal colour fundus images. We trained our model on 76 370 retinal fundus images from 13 099 patients with diabetes who had participated in the Singapore Integrated Diabetic Retinopathy Program, between 2010 and 2013, which has been published previously. In this clinical validation study, we included all patients with a diagnosis of diabetes that attended a mobile screening unit in five urban centres in the Copperbelt province of Zambia from Feb 1 to June 31, 2012. In our model, referable diabetic retinopathy was defined as moderate non-proliferative diabetic retinopathy or worse, diabetic macular oedema, and ungradable images. Vision-threatening diabetic retinopathy comprised severe non-proliferative and proliferative diabetic retinopathy. We calculated the area under the curve (AUC), sensitivity, and specificity for referable diabetic retinopathy, and sensitivities of vision-threatening diabetic retinopathy and diabetic macular oedema compared with the grading by retinal specialists. We did a multivariate analysis for systemic risk factors and referable diabetic retinopathy between AI and human graders. FINDINGS A total of 4504 retinal fundus images from 3093 eyes of 1574 Zambians with diabetes were prospectively recruited. Referable diabetic retinopathy was found in 697 (22·5%) eyes, vision-threatening diabetic retinopathy in 171 (5·5%) eyes, and diabetic macular oedema in 249 (8·1%) eyes. The AUC of the AI system for referable diabetic retinopathy was 0·973 (95% CI 0·969-0·978), with corresponding sensitivity of 92·25% (90·10-94·12) and specificity of 89·04% (87·85-90·28). Vision-threatening diabetic retinopathy sensitivity was 99·42% (99·15-99·68) and diabetic macular oedema sensitivity was 97·19% (96·61-97·77). The AI model and human graders showed similar outcomes in referable diabetic retinopathy prevalence detection and systemic risk factors associations. Both the AI model and human graders identified longer duration of diabetes, higher level of glycated haemoglobin, and increased systolic blood pressure as risk factors associated with referable diabetic retinopathy. INTERPRETATION An AI system shows clinically acceptable performance in detecting referable diabetic retinopathy, vision-threatening diabetic retinopathy, and diabetic macular oedema in population-based diabetic retinopathy screening. This shows the potential application and adoption of such AI technology in an under-resourced African population to reduce the incidence of preventable blindness, even when the model is trained in a different population. FUNDING National Medical Research Council Health Service Research Grant, Large Collaborative Grant, Ministry of Health, Singapore; the SingHealth Foundation; and the Tanoto Foundation.

[1]  M. García-Fiñana,et al.  First Prospective Cohort Study of Diabetic Retinopathy from Sub-Saharan Africa , 2016, Ophthalmology.

[2]  Ankur Taly,et al.  Axiomatic Attribution for Deep Networks , 2017, ICML.

[3]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[4]  Daniel Shu Wei Ting MMed,et al.  Diabetic retinopathy: global prevalence, major risk factors, screening practices and public health challenges: a review , 2016 .

[5]  M. He,et al.  Efficacy of a Deep Learning System for Detecting Glaucomatous Optic Neuropathy Based on Color Fundus Photographs. , 2018, Ophthalmology.

[6]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[7]  M. Abràmoff,et al.  Results of Automated Retinal Image Analysis for Detection of Diabetic Retinopathy from the Nakuru Study, Kenya , 2015, PloS one.

[8]  Sobha Sivaprasad,et al.  Prevalence of diabetic retinopathy in various ethnic groups: a worldwide perspective. , 2012, Survey of ophthalmology.

[9]  Hans Limburg,et al.  Global causes of blindness and distance vision impairment 1990-2020: a systematic review and meta-analysis. , 2017, The Lancet. Global health.

[10]  M. Abràmoff,et al.  Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep Learning. , 2016, Investigative ophthalmology & visual science.

[11]  S. Harding,et al.  Epidemiology of diabetic retinopathy and maculopathy in Africa: a systematic review , 2013, Diabetic medicine : a journal of the British Diabetic Association.

[12]  E. Finkelstein,et al.  Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes , 2017, JAMA.

[13]  Hans Limburg,et al.  Prevalence and causes of blindness and vision impairment: magnitude, temporal trends and projections in South and Central Asia , 2018, British Journal of Ophthalmology.

[14]  M. García-Fiñana,et al.  Incidence and progression of diabetic retinopathy in Sub-Saharan Africa: A five year cohort study , 2017, PloS one.

[15]  T. Peto,et al.  Prevalence and Correlates of Diabetic Retinopathy in a Population-based Survey of Older People in Nakuru, Kenya , 2014, Ophthalmic epidemiology.

[16]  Daniel S W Ting,et al.  Clinical Applicability of Deep Learning System in Detecting Tuberculosis with Chest Radiography. , 2018, Radiology.

[17]  V. Hall,et al.  Diabetes in Sub Saharan Africa 1999-2011: Epidemiology and public health implications. a systematic review , 2011, BMC public health.

[18]  B. Klein,et al.  Global Prevalence and Major Risk Factors of Diabetic Retinopathy , 2012, Diabetes Care.

[19]  D. Hood,et al.  Efficacy of a Deep Learning System for Detecting Glaucomatous Optic Neuropathy Based on Color Fundus Photographs. , 2018, Ophthalmology.

[20]  Jennifer K. Sun,et al.  Guidelines on Diabetic Eye Care: The International Council of Ophthalmology Recommendations for Screening, Follow-up, Referral, and Treatment Based on Resource Settings. , 2018, Ophthalmology.

[21]  H. Doctor,et al.  Health facility delivery in sub-Saharan Africa: successes, challenges, and implications for the 2030 development agenda , 2018, BMC Public Health.

[22]  M. Abràmoff,et al.  Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices , 2018, npj Digital Medicine.

[23]  S. Resnikoff,et al.  The number of ophthalmologists in practice and training worldwide: a growing gap despite more than 200 000 practitioners , 2012, British Journal of Ophthalmology.

[24]  S. Sivaprasad,et al.  Prevalence of diabetic retinopathy and visual impairment in patients with diabetes mellitus in Zambia through the implementation of a mobile diabetic retinopathy screening project in the Copperbelt province: a cross-sectional study , 2018, Eye.

[25]  Michael V. McConnell,et al.  Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning , 2017, Nature Biomedical Engineering.

[26]  Neil J. Joshi,et al.  Automated Grading of Age-Related Macular Degeneration From Color Fundus Images Using Deep Convolutional Neural Networks , 2017, JAMA ophthalmology.

[27]  Neil J. Joshi,et al.  Use of Deep Learning for Detailed Severity Characterization and Estimation of 5-Year Risk Among Patients With Age-Related Macular Degeneration , 2018, JAMA ophthalmology.

[28]  Pei Ying Lee,et al.  An Automated Grading System for Detection of Vision-Threatening Referable Diabetic Retinopathy on the Basis of Color Fundus Photographs , 2018, Diabetes Care.

[29]  Rishab Gargeya,et al.  Automated Identification of Diabetic Retinopathy Using Deep Learning. , 2017, Ophthalmology.

[30]  James M. Brown,et al.  Automated Diagnosis of Plus Disease in Retinopathy of Prematurity Using Deep Convolutional Neural Networks , 2018, JAMA ophthalmology.

[31]  A. Motala,et al.  Diabetes in the Africa Region: an update. , 2014, Diabetes research and clinical practice.

[32]  T. Peto,et al.  The incidence of diabetes mellitus and diabetic retinopathy in a population-based cohort study of people age 50 years and over in Nakuru, Kenya , 2017, BMC Endocrine Disorders.