CycleGAN-based deep learning technique for artifact reduction in fundus photography

Purpose A low quality of fundus photograph with artifacts may lead to false diagnosis. Recently, a cycle-consistent generative adversarial network (CycleGAN) has been introduced as a tool to generate images without matching paired images. Therefore, herein, we present a deep learning technique that removes the artifacts automatically in a fundus photograph using a CycleGAN model. Methods This study included a total of 2206 anonymized retinal images including 1146 with artifacts and 1060 without artifacts. In this experiment, we applied the CycleGAN model to color fundus photographs with a pixel resolution of 256 × 256 × 3. To apply the CycleGAN to an independent dataset, we randomly divided the data into training (90%) and test (10%) datasets. Additionally, we adopted the automated quality evaluation (AQE) to assess the retinal image quality. Results From the results, we observed that the artifacts such as overall haze, edge haze, lashes, arcs, and uneven illumination were successfully reduced by the CycleGAN in the generated images, and the main information of the retina was essentially retained. Further, we observed that most of the generated images exhibited improved AQE grade values when compared with the original images with artifacts. Conclusion Thus, we could conclude that the CycleGAN technique can effectively reduce the artifacts and improve the quality of fundus photographs, and it may be beneficial for clinicians in analyzing the low-quality fundus photographs. Future studies should improve the quality and resolution of the generated image to provide a more detailed fundus photography.

[1]  Bin Yan,et al.  Unpaired Low-Dose CT Denoising Network Based on Cycle-Consistent Generative Adversarial Network with Prior Image Information , 2019, Comput. Math. Methods Medicine.

[2]  Jung Sub Kim,et al.  Adopting machine learning to automatically identify candidate patients for corneal refractive surgery , 2019, npj Digital Medicine.

[3]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[4]  Tae Keun Yoo,et al.  A generative adversarial network approach to predicting postoperative appearance after orbital decompression surgery for thyroid eye disease , 2020, Comput. Biol. Medicine.

[5]  Deok Won Kim,et al.  The possibility of the combination of OCT and fundus images for improving the diagnostic accuracy of deep learning for age-related macular degeneration: a preliminary experiment , 2018, Medical & Biological Engineering & Computing.

[6]  José Manuel Fonseca,et al.  A template matching technique for artifacts detection in retinal images , 2013, 2013 8th International Symposium on Image and Signal Processing and Analysis (ISPA).

[7]  A. Alm,et al.  The Glaucoma Guidelines of the Swedish Ophthalmological Society , 2012, Acta ophthalmologica. Supplement.

[8]  Kp Suresh.  An overview of randomization techniques: An unbiased assessment of outcome in clinical research , 2011, Journal of human reproductive sciences.

[9]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[10]  Lin Yang,et al.  Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Jianxin Shen,et al.  Fundus Image Enhancement Method Based on CycleGAN , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[12]  Yue Yu,et al.  SAR-to-Optical Image Translation Using Supervised Cycle-Consistent Adversarial Networks , 2019, IEEE Access.

[13]  Marco Stampanoni,et al.  Unsupervised data to content transformation with histogram-matching cycle-consistent generative adversarial networks , 2019, Nature Machine Intelligence.

[14]  Stan Z. Li,et al.  Face Image Quality Evaluation for ISO/IEC Standards 19794-5 and 29794-5 , 2009, ICB.

[15]  Peter Wanger,et al.  Automated quality evaluation of digital fundus photographs , 2009, Acta ophthalmologica.

[16]  Neil J. Joshi,et al.  Assessment of Deep Generative Models for High-Resolution Synthetic Retinal Image Generation of Age-Related Macular Degeneration , 2019, JAMA ophthalmology.

[17]  Lovedeep Gondara,et al.  Medical Image Denoising Using Convolutional Denoising Autoencoders , 2016, 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW).

[18]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Filip Sroubek,et al.  Retinal image restoration by means of blind deconvolution. , 2011, Journal of biomedical optics.

[20]  Joachim Hornegger,et al.  Quality-Guided Denoising for Low-Cost Fundus Imaging , 2012, Bildverarbeitung für die Medizin.

[21]  J. Olson,et al.  Automated assessment of diabetic retinal image quality based on clarity and field definition. , 2006, Investigative ophthalmology & visual science.

[22]  Ender Konukoglu,et al.  Injecting and removing suspicious features in breast imaging with CycleGAN: A pilot study of automated adversarial attacks using neural networks on small images. , 2019, European journal of radiology.

[23]  Ulas Bagci,et al.  Cross-modality Knowledge Transfer for Prostate Segmentation from CT Scans , 2019, DART/MIL3ID@MICCAI.

[24]  Dahong Qian,et al.  Human Visual System-Based Fundus Image Quality Assessment of Portable Fundus Camera Photographs , 2016, IEEE Transactions on Medical Imaging.

[25]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[26]  Yu Liu,et al.  An Extensive Study of Cycle-Consistent Generative Networks for Image-to-Image Translation , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[27]  Gui-Bin Bian,et al.  Performance Analysis of Google Colaboratory as a Tool for Accelerating Deep Learning Applications , 2018, IEEE Access.

[28]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[29]  Sang Jun Park,et al.  Towards Accurate Segmentation of Retinal Vessels and the Optic Disc in Fundoscopic Images with Generative Adversarial Networks , 2018, Journal of Digital Imaging.