The application of deep learning framework in quantifying retinal structures on ophthalmic image in research eye-PACS

The rise of deep learning (DL) framework and its application in object recognition could benefit image-based medical diagnosis. Since eye is believed to be a window into human health, the application of DL on differentiating abnormal ophthalmic photography (OP) will greatly empower ophthalmologists to relieve their workload for disease screening. In our previous work, we employed ResNet-50 to construct classification model for diabetic retinopathy(DR) within the PACS. In this study, we implemented latest DL object detection and semantic segmentation framework to empower the eye-PACS. Mask R-CNN framework was selected for object detection and instance segmentation of the optic disc (OD) and the macula. Furthermore, Unet framework was utilized for semantic segmentation of retinal vessel pixels from OP. The performance of the segmented results by two frameworks achieved state-of-art efficiency and the segmented results were transmitted to PACS as grayscale softcopy presentation state (GSPS) file. We also developed a prototype for OP quantitative analysis. It’s believed that the implementation of DL framework into the object recognition and analysis on OPs is meaningful and worth further investigation.

[1]  Ling Luo,et al.  Retinal blood vessels semantic segmentation method based on modified U-Net , 2018, 2018 Chinese Control And Decision Conference (CCDC).

[2]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Xiaogang Li,et al.  Convolutional neural networks based transfer learning for diabetic retinopathy fundus image classification , 2017, 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[4]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[6]  Xiaoyi Jiang,et al.  Adaptive Local Thresholding by Verification-Based Multithreshold Probing with Application to Vessel Detection in Retinal Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Siliang Zhang,et al.  The development of an ophthalmologic imaging CADe structured report for retinal image radiomics research , 2018, Medical Imaging.

[8]  Amit Kale,et al.  Automatic Optic Disk and Cup Segmentation of Fundus Images Using Deep Learning , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[9]  Siliang Zhang,et al.  The application of deep learning for diabetic retinopathy prescreening in research eye-PACS , 2018, Medical Imaging.

[10]  Elisa Ricci,et al.  Retinal Blood Vessel Segmentation Using Line Operators and Support Vector Classification , 2007, IEEE Transactions on Medical Imaging.

[11]  Sang Jun Park,et al.  Retinal Vessel Segmentation in Fundoscopic Images with Generative Adversarial Networks , 2017, ArXiv.

[12]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[13]  Emanuele Trucco,et al.  Retinal vessel segmentation using multiwavelet kernels and multiscale hierarchical decomposition , 2013, Pattern Recognit..

[14]  Lei Zhang,et al.  Retinal vessel extraction by matched filter with first-order derivative of Gaussian , 2010, Comput. Biol. Medicine.

[15]  István Csabai,et al.  Detecting and classifying lesions in mammograms with Deep Learning , 2017, Scientific Reports.