Automated identification of retinopathy of prematurity by image-based deep learning

Background Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide but can be a treatable retinal disease with appropriate and timely diagnosis. This study was performed to develop a robust intelligent system based on deep learning to automatically classify the severity of ROP from fundus images and detect the stage of ROP and presence of plus disease to enable automated diagnosis and further treatment. Methods A total of 36,231 fundus images were labeled by 13 licensed retinal experts. A 101-layer convolutional neural network (ResNet) and a faster region-based convolutional neural network (Faster-RCNN) were trained for image classification and identification. We applied a 10-fold cross-validation method to train and optimize our algorithms. The accuracy, sensitivity, and specificity were assessed in a four-degree classification task to evaluate the performance of the intelligent system. The performance of the system was compared with results obtained by two retinal experts. Moreover, the system was designed to detect the stage of ROP and presence of plus disease as well as to highlight lesion regions based on an object detection network using Faster-RCNN. Results The system achieved an accuracy of 0.903 for the ROP severity classification. Specifically, the accuracies in discriminating normal, mild, semi-urgent, and urgent were 0.883, 0.900, 0.957, and 0.870, respectively; the corresponding accuracies of the two experts were 0.902 and 0.898. Furthermore, our model achieved an accuracy of 0.957 for detecting the stage of ROP and 0.896 for detecting plus disease; the accuracies in discriminating stage I to stage V were 0.876, 0.942, 0.968, 0.998 and 0.999, respectively. Conclusions Our system was able to detect ROP and differentiate four-level classification fundus images with high accuracy and specificity. The performance of the system was comparable to or better than that of human experts, demonstrating that this system could be used to support clinical decisions.

[1]  L. Peng,et al.  Deep learning in ophthalmology: The technical and clinical considerations , 2019, Progress in Retinal and Eye Research.

[2]  James M. Brown,et al.  Automated Diagnosis of Plus Disease in Retinopathy of Prematurity Using Deep Convolutional Neural Networks , 2018, JAMA ophthalmology.

[3]  Bram van Ginneken,et al.  Fifty years of computer analysis in chest imaging: rule-based, machine learning, deep learning , 2017, Radiological Physics and Technology.

[4]  Gabriel J. Brostow,et al.  Automated Retinopathy of Prematurity Case Detection with Convolutional Neural Networks , 2016, LABELS/DLMIA@MICCAI.

[5]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[6]  Jian Sun,et al.  Convolutional neural networks at constrained time cost , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[8]  Nico Karssemeijer,et al.  Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images , 2017, Journal of medical imaging.

[10]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  W. Tasman,et al.  Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial. , 2004, Archives of ophthalmology.

[12]  A. Fielder,et al.  Preliminary results of treatment of eyes with high-risk prethreshold retinopathy of prematurity in the early treatment for retinopathy of prematurity randomized trial. , 2003, Archives of ophthalmology.

[13]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[14]  Eric Eaton,et al.  Artificial Intelligence for Pediatric Ophthalmology , 2019, Current opinion in ophthalmology.

[15]  Deniz Erdoğmuş,et al.  Computer-Based Image Analysis for Plus Disease Diagnosis in Retinopathy of Prematurity: Performance of the "i-ROP" System and Image Features Associated With Expert Diagnosis. , 2015, Translational vision science & technology.

[16]  A. Fielder,et al.  Preterm-associated visual impairment and estimates of retinopathy of prematurity at regional and global levels for 2010 , 2013, Pediatric Research.

[17]  W. Göpel,et al.  Screening and Treatment in Retinopathy of Prematurity. , 2015, Deutsches Arzteblatt international.

[18]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[19]  Alan L Robin,et al.  Challenges of ophthalmic care in the developing world. , 2014, JAMA ophthalmology.

[20]  Anna L. Ells,et al.  The International Classification of Retinopathy of Prematurity revisited. , 2005, Archives of ophthalmology.

[21]  Alexandre Hoang Thiery,et al.  Glaucoma management in the era of artificial intelligence , 2019, British Journal of Ophthalmology.

[22]  Daniel L. Rubin,et al.  Automated Grading of Gliomas using Deep Learning in Digital Pathology Images: A modular approach with ensemble of convolutional neural networks , 2015, AMIA.

[23]  Zhang Yi,et al.  Automated Analysis for Retinopathy of Prematurity by Deep Neural Networks , 2019, IEEE Transactions on Medical Imaging.

[24]  Verónica Bolón-Canedo,et al.  Dealing with inter-expert variability in retinopathy of prematurity: A machine learning approach , 2015, Comput. Methods Programs Biomed..

[25]  Michael F Chiang,et al.  Plus disease in retinopathy of prematurity: pilot study of computer-based and expert diagnosis. , 2007, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[26]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[27]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[28]  N. Patton,et al.  Digital image analysis of plus disease in retinopathy of prematurity , 2009, Acta ophthalmologica.

[29]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[31]  D Erdogmus,et al.  Analysis of Underlying Causes of Inter-expert Disagreement in Retinopathy of Prematurity Diagnosis , 2014, Methods of Information in Medicine.

[32]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Zhang Yi,et al.  Automated retinopathy of prematurity screening using deep neural networks , 2018, EBioMedicine.

[34]  Michael F Chiang,et al.  Interexpert agreement in the identification of macular location in infants at risk for retinopathy of prematurity. , 2010, Archives of ophthalmology.

[35]  Amir Sadeghipour,et al.  Artificial intelligence in retina , 2018, Progress in Retinal and Eye Research.

[36]  Deniz Erdogmus,et al.  Expert Diagnosis of Plus Disease in Retinopathy of Prematurity From Computer-Based Image Analysis. , 2016, JAMA ophthalmology.

[37]  D. Wallace,et al.  A pilot study using "ROPtool" to quantify plus disease in retinopathy of prematurity. , 2007, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[38]  M. Chiang,et al.  Computer-based image analysis for plus disease diagnosis in retinopathy of prematurity. , 2012, Journal of pediatric ophthalmology and strabismus.