Dehaze of Cataractous Retinal Images Using an Unpaired Generative Adversarial Network

Cataracts are the leading cause of visual impairment worldwide. Examination of the retina through cataracts using a fundus camera is challenging and error-prone due to degraded image quality. We sought to develop an algorithm to dehaze such images to support diagnosis by either ophthalmologists or computer-aided diagnosis systems. Based on the generative adversarial network (GAN) concept, we designed two neural networks: CataractSimGAN and CataractDehazeNet. CataractSimGAN was intended for the synthesis of cataract-like images through unpaired clear retinal images and cataract images. CataractDehazeNet was trained using pairs of synthesized cataract-like images and the corresponding clear images through supervised learning. With two networks trained independently, the number of hyper-parameters was reduced, leading to better performance. We collected 400 retinal images without cataracts and 400 hazy images from cataract patients as the training dataset. Fifty cataract images and the corresponding clear images from the same patients after surgery comprised the test dataset. The clear images after surgery were used for reference to evaluate the performance of our method. CataractDehazeNet was able to enhance the degraded image from cataract patients substantially and to visualize blood vessels and the optic disc, while actively suppressing the artifacts common in application of similar methods. Thus, we developed an algorithm to improve the quality of the retinal images acquired from cataract patients. We achieved high structure similarity and fidelity between processed images and images from the same patients after cataract surgery.

[1]  Zhiwei Wang,et al.  Bi-Modality Medical Image Synthesis Using Semi-Supervised Sequential Generative Adversarial Networks , 2020, IEEE Journal of Biomedical and Health Informatics.

[2]  Tomas Pfister,et al.  Learning from Simulated and Unsupervised Images through Adversarial Training , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[4]  Rong Chen,et al.  Single Image Haze Removal via Region Detection Network , 2019, IEEE Transactions on Multimedia.

[5]  Oerip S. Santoso,et al.  Color retinal image enhancement using CLAHE , 2013, International Conference on ICT for Smart Society.

[6]  Robby T. Tan,et al.  Visibility in bad weather from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Delia Cabrera DeBuc,et al.  Deep Learning based Retinal OCT Segmentation , 2018, Comput. Biol. Medicine.

[8]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[9]  Subrahmanyam Murala,et al.  RI-GAN: An End-To-End Network for Single Image Haze Removal , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[11]  Ling Shao,et al.  Understanding and Correcting Low-quality Retinal Fundus Images for Clinical Analysis , 2020, ArXiv.

[12]  Mohinder Malhotra Single Image Haze Removal Using Dark Channel Prior , 2016 .

[13]  Jianqiang Li,et al.  A computer-aided healthcare system for cataract classification and grading based on fundus image analysis , 2015, Comput. Ind..

[14]  Huiqi Li,et al.  Data-Driven Enhancement of Blurry Retinal Images via Generative Adversarial Networks , 2019, MICCAI.

[15]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[16]  Gilberto Zamora,et al.  Automated image quality evaluation of retinal fundus photographs in diabetic retinopathy screening , 2012, 2012 IEEE Southwest Symposium on Image Analysis and Interpretation.

[17]  Stephen Lin,et al.  Angle-Closure Detection in Anterior Segment OCT Based on Multilevel Deep Network , 2019, IEEE Transactions on Cybernetics.

[18]  Hazim Kemal Ekenel,et al.  Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[19]  Mingzhai Sun,et al.  Automated diagnosis and quantitative analysis of plus disease in retinopathy of prematurity based on deep convolutional neural networks , 2019, Acta ophthalmologica.

[20]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[22]  Rui Zheng,et al.  Simultaneous Arteriole and Venule Segmentation of Dual-Modal Fundus Images Using a Multi-Task Cascade Network , 2019, IEEE Access.

[23]  Shenghua Gao,et al.  Noise Adaptation Generative Adversarial Network for Medical Image Analysis , 2020, IEEE Transactions on Medical Imaging.

[24]  Xi Chen,et al.  Unsupervised Deep Learning Features for Lung Cancer Overall Survival Analysis , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[25]  Stephen Lin,et al.  A Deep Learning System for Automated Angle-Closure Detection in Anterior Segment Optical Coherence Tomography Images. , 2019, American journal of ophthalmology.

[26]  Ling Shao,et al.  Human-Aware Motion Deblurring , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[27]  Jack T. Holladay,et al.  Refraction and clinical optics , 1981 .

[28]  E Peli,et al.  Restoration of retinal images obtained through cataracts. , 1989, IEEE transactions on medical imaging.

[29]  R. D. Ferguson,et al.  Adaptive optics scanning laser ophthalmoscope for stabilized retinal imaging. , 2006, Optics express.

[30]  Shenghua Gao,et al.  SkrGAN: Sketching-rendering Unconditional Generative Adversarial Networks for Medical Image Synthesis , 2019, MICCAI.

[31]  Huazhu Fu,et al.  Evaluation of Retinal Image Quality Assessment Networks in Different Color-spaces , 2019, MICCAI.

[32]  Mei Zhou,et al.  Color Retinal Image Enhancement Based on Luminosity and Contrast Adjustment , 2018, IEEE Transactions on Biomedical Engineering.

[33]  Shree K. Nayar,et al.  Vision and the Atmosphere , 2002, International Journal of Computer Vision.

[34]  Xiaochun Cao,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks , 2016, ECCV.

[35]  Bin Li,et al.  Detection of exudates in fundus photographs with imbalanced learning using conditional generative adversarial network. , 2018, Biomedical optics express.

[36]  Zhao Liu,et al.  A fast method for single image haze removal based on multiscale dark channel prior , 2018, International Symposium on Multispectral Image Processing and Pattern Recognition.

[37]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Zhengguo Li,et al.  Structure-Preserving Guided Retinal Image Filtering and Its Application for Optic Disk Analysis , 2018, IEEE Transactions on Medical Imaging.

[39]  T. Hebert,et al.  Adaptive optics scanning laser ophthalmoscopy. , 2002, Optics express.

[40]  D. Pascolini,et al.  Global estimates of visual impairment: 2010 , 2011, British Journal of Ophthalmology.

[41]  R. Webb,et al.  Confocal scanning laser ophthalmoscope. , 1987, Applied optics.

[42]  John E. Hopcroft,et al.  Stacked Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Cheng-Hsiung Hsieh,et al.  Single Image Haze Removal Using Weak Dark Channel Prior , 2018, 2018 9th International Conference on Awareness Science and Technology (iCAST).

[44]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[45]  Raanan Fattal,et al.  Single image dehazing , 2008, ACM Trans. Graph..

[46]  Per Fagerholm,et al.  [Age-related cataract]. , 2006, Lakartidningen.

[47]  Subrahmanyam Murala,et al.  C^2MSNet: A Novel Approach for Single Image Haze Removal , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[48]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[49]  F. Delori,et al.  Scanning Laser Ophthalmoscopy (SLO) , 2019, High Resolution Imaging in Microscopy and Ophthalmology.

[50]  Hang Li,et al.  Dense Deconvolutional Network for Skin Lesion Segmentation , 2019, IEEE Journal of Biomedical and Health Informatics.

[51]  Anil Kumar Tiwari,et al.  Fundus image quality assessment: survey, challenges, and future scope , 2019, IET Image Process..

[52]  Jianqiang Li,et al.  A Hybrid Global-Local Representation CNN Model for Automatic Cataract Grading , 2020, IEEE Journal of Biomedical and Health Informatics.

[53]  Jan Kautz,et al.  Deep Semantic Face Deblurring , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.