A Deep Learning-based Quality Assessment and Segmentation System with a Large-scale Benchmark Dataset for Optical Coherence Tomographic Angiography Image

Optical Coherence Tomography Angiography (OCTA) is a non-invasive and non-contacting imaging technique providing visualization of microvasculature of retina and optic nerve head in human eyes in vivo. The adequate image quality of OCTA is the prerequisite for the subsequent quantification of retinal microvasculature. Traditionally, the image quality score based on signal strength is used for discriminating low quality. However, it is insufficient for identifying artefacts such as motion and off-centration, which rely specialized knowledge and need tedious and time-consuming manual identification. One of the most primary issues in OCTA analysis is to sort out the foveal avascular zone (FAZ) region in the retina, which highly correlates with any visual acuity disease. However, the variations in OCTA visual quality affect the performance of deep learning in any downstream marginally. Moreover, filter*Correspondance to W. Cheng and W. Wang. Y. Wang: yufei8828@gmail.com, Y. Shen: shenyq@sjtu.edu.cn, M. Yuan: dr.yuanm@qq.com, J. Xu: xu-j20@mails.tsinghua.edu.cn, B. Yang: yangbin8188@qq.com, C. Liu: chi.liu@student.uts.edu.au, W. Cai: cherylwenjia@outlook.com, W. Cheng: 18120798001@163.com, W. Wang: zoc wangwei@yahoo.com. ing the low-quality OCTA images out is both labor-intensive and time-consuming. To address these issues, we develop an automated computer-aided OCTA image processing system using deep neural networks as the classifier and segmentor to help ophthalmologists in clinical diagnosis and research. This system can be an assistive tool as it can process OCTA images of different formats to assess the quality and segment the FAZ area. The source code is freely available at https://github.com/shanzha09/COIPS.git. Another major contribution is the large-scale OCTA dataset, namely OCTA-25K-IQA-SEG we publicize for performance evaluation. It is comprised of four subsets, namely sOCTA-3×3-10k, sOCTA-6×6-14k, sOCTA-3×31.1k-seg, and dOCTA-6×6-1.1k-seg, which contains a total number of 25,665 images. The large-scale OCTA dataset is available at https://doi.org/10.5281/zenodo. 5111975, https://doi.org/10.5281/zenodo. 5111972.

[1]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[2]  Bryan M. Williams,et al.  A Novel Deep Learning Based OCTA De-striping Method , 2019, MIUA.

[3]  Jonathan Cheung-Wai Chan,et al.  Learning and Transferring Deep Joint Spectral–Spatial Features for Hyperspectral Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Qinghua Zhao,et al.  Brain tumor classification for MR images using transfer learning and fine-tuning , 2019, Comput. Medical Imaging Graph..

[5]  Tom MacGillivray,et al.  Automated Segmentation of Optical Coherence Tomography Angiography Images: Benchmark Data and Clinically Relevant Metrics , 2019, Translational vision science & technology.

[6]  J L Lauermann,et al.  Automated OCT angiography image quality assessment using a deep learning algorithm , 2019, Graefe's Archive for Clinical and Experimental Ophthalmology.

[7]  Frank Hutter,et al.  Fixing Weight Decay Regularization in Adam , 2017, ArXiv.

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Yali Jia,et al.  Development and validation of a deep learning algorithm for distinguishing the nonperfusion area from signal reduction artifacts on OCT angiography. , 2019, Biomedical optics express.

[10]  David Huang,et al.  MEDnet, a neural network for automated detection of avascular area in OCT angiography. , 2018, Biomedical optics express.

[11]  Nima Tajbakhsh,et al.  UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation , 2020, IEEE Transactions on Medical Imaging.

[12]  Hirokazu Ishii,et al.  Automated Measurement of the Foveal Avascular Zone in Swept-Source Optical Coherence Tomography Angiography Images , 2019, Translational vision science & technology.

[13]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  L. Yannuzzi,et al.  Visual Acuity Is Correlated with the Area of the Foveal Avascular Zone in Diabetic Retinopathy and Retinal Vein Occlusion. , 2016, Ophthalmology.

[15]  C. Cheung,et al.  Reliability of foveal avascular zone metrics automatically measured by Cirrus optical coherence tomography angiography in healthy subjects , 2019, International Ophthalmology.

[16]  Adnan Saood.  COVID-19 lung CT image segmentation using deep learning methods: U-Net versus SegNet , 2021, BMC Medical Imaging.

[17]  James S. Duncan,et al.  Anatomy-guided Multimodal Registration by Learning Segmentation without Ground Truth: Application to Intraprocedural CBCT/MR Liver Segmentation and Registration , 2021, Medical Image Anal..

[18]  Jing Ke,et al.  Sampling Based Tumor Recognition in Whole-Slide Histology Image With Deep Learning Approaches , 2021, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[19]  C. Shields,et al.  IMAGE QUALITY AND ARTIFACTS ON OPTICAL COHERENCE TOMOGRAPHY ANGIOGRAPHY: Comparison of Pathologic and Paired Fellow Eyes in 65 Patients With Unilateral Choroidal Melanoma Treated With Plaque Radiotherapy , 2017, Retina.

[20]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[21]  Automated quantification of superficial retinal capillaries and large vessels for diabetic retinopathy on optical coherence tomographic angiography , 2019, Journal of biophotonics.

[22]  Zhen Zhao,et al.  Automated diagnosis of bone metastasis based on multi-view bone scans using attention-augmented deep neural networks , 2020, Medical Image Anal..

[23]  Jason Hsu,et al.  Quantification of Diabetic Macular Ischemia Using Optical Coherence Tomography Angiography and Its Relationship with Visual Acuity. , 2017, Ophthalmology.

[24]  J. Jonas,et al.  Count and density of human retinal photoreceptors , 2004, Graefe's Archive for Clinical and Experimental Ophthalmology.

[25]  Rachel E Linderman,et al.  Earliest Evidence of Preclinical Diabetic Retinopathy Revealed Using Optical Coherence Tomography Angiography Perfused Capillary Density. , 2019, American journal of ophthalmology.

[26]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Thomas Brox,et al.  U-Net: deep learning for cell counting, detection, and morphometry , 2018, Nature Methods.

[28]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[29]  Frank Hutter,et al.  SGDR: Stochastic Gradient Descent with Warm Restarts , 2016, ICLR.

[30]  Aki Kato,et al.  ENLARGEMENT OF FOVEAL AVASCULAR ZONE IN DIABETIC EYES EVALUATED BY EN FACE OPTICAL COHERENCE TOMOGRAPHY ANGIOGRAPHY , 2015, Retina.

[31]  Joel J. P. C. Rodrigues,et al.  A novel deep learning based framework for the detection and classification of breast cancer using transfer learning , 2019, Pattern Recognit. Lett..

[32]  Dao-Yi Yu,et al.  Intraretinal oxygen distribution in the monkey retina and the response to systemic hyperoxia. , 2005, Investigative ophthalmology & visual science.

[33]  Sandor R. Ferenczy,et al.  VARIABILITY IN FOVEAL AVASCULAR ZONE AND CAPILLARY DENSITY USING OPTICAL COHERENCE TOMOGRAPHY ANGIOGRAPHY MACHINES IN HEALTHY EYES , 2017, Retina.

[34]  Club Jules Gonin,et al.  Graefe's archive for clinical and experimental ophthalmology , 1982 .

[35]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[36]  K. Falavarjani,et al.  Foveal avascular zone segmentation in optical coherence tomography angiography images using a deep learning approach , 2021, Scientific Reports.

[37]  C. Cheung,et al.  Improved Automated Foveal Avascular Zone Measurement in Cirrus Optical Coherence Tomography Angiography Using the Level Sets Macro , 2020, Translational vision science & technology.

[38]  Manuel Casselholmde Salles,et al.  Optical Coherence Tomography Angiography in Central Retinal Vein Occlusion: Correlation Between the Foveal Avascular Zone and Visual Acuity. , 2016, Investigative ophthalmology & visual science.

[39]  R. Spaide,et al.  Retinal vascular layers imaged by fluorescein angiography and optical coherence tomography angiography. , 2015, JAMA ophthalmology.

[40]  Rama Chellappa,et al.  HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Xincheng Yao,et al.  Supervised Machine Learning Based Multi-Task Artificial Intelligence Classification of Retinopathies , 2019, Journal of clinical medicine.

[42]  Jing Ke,et al.  Representative Region Based Active Learning For Histological Classification Of Colorectal Cancer , 2021, 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI).

[43]  Stephen Lin,et al.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[44]  Jost L. Lauermann,et al.  Optical Coherence Tomography Angiography Offers New Insights into Choriocapillaris Perfusion , 2018, Ophthalmologica.

[45]  Jeong W. Pak,et al.  Prevalence and Severity of Artifacts in Optical Coherence Tomographic Angiograms. , 2019, JAMA ophthalmology.

[46]  Hui Xiong,et al.  A Comprehensive Survey on Transfer Learning , 2019, Proceedings of the IEEE.

[47]  Xin Zhang,et al.  HF-UNet: Learning Hierarchically Inter-Task Relevance in Multi-Task U-Net for Accurate Prostate Segmentation in CT Images , 2020, IEEE Transactions on Medical Imaging.

[48]  D. Sarraf,et al.  Optical coherence tomography angiography of the retina and choroid; current applications and future directions , 2017, Journal of current ophthalmology.

[49]  Léon Bottou,et al.  Stochastic Gradient Descent Tricks , 2012, Neural Networks: Tricks of the Trade.

[50]  Siping Chen,et al.  Can deep learning improve the automatic segmentation of deep foveal avascular zone in optical coherence tomography angiography? , 2021, Biomed. Signal Process. Control..

[51]  T. Peto,et al.  Wide-field imaging and OCT vs clinical evaluation of patients referred from diabetic retinopathy screening , 2015, Eye.

[52]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.