CGAN-based Synthetic Medical Image Augmentation between Retinal Fundus Images and Vessel Segmented Images

Computer-aided medical diagnosis is widely used to assist in the interpretation of medical images. Early screening of diseases has a great value for the clinical diagnosis. However, medical images are scarce and fraught with legal concerns about patient privacy. Inspired by the prevailing learning capability of Generative Adversarial Network (GAN), We propose simulated learning: it first learns a mapping from retinal fundus images to vessel segmented images and then trains a learning model to generate synthetic fundus images. In the results, we evaluated our proposed method from both quantitative and qualitative aspects. For generated segmented images, our method achieves a dice coefficient of 0.838 and an AUC of 0.985 on the DRIVE datasets. You can also see more information about our medical image algorithm results at https://github.com/qiqi7788/Bidirectional-Mapping-formedical-image. These results also verify the effectiveness of our algorithm.

[1]  Victor S. Lempitsky,et al.  N4-Fields: Neural Network Nearest Neighbor Fields for Image Transforms , 2014, ArXiv.

[2]  Huiqi Li,et al.  Synthesizing retinal and neuronal images with generative adversarial nets , 2018, Medical Image Anal..

[3]  R. Howe,et al.  17th International Conference on Medical Image Computing and Computer-Assisted Intervention. , 2014, Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention.

[4]  Américo Oliveira,et al.  Retinal vessel segmentation based on Fully Convolutional Neural Networks , 2018, Expert Syst. Appl..

[5]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[6]  Ling Luo,et al.  Retinal blood vessels semantic segmentation method based on modified U-Net , 2018, 2018 Chinese Control And Decision Conference (CCDC).

[7]  Yiheng Cai,et al.  Retinal blood vessel segmentation based on the Gaussian matched filter and U-net , 2017, 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[8]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[9]  James M. Brown,et al.  Semi-Supervised Deep Learning for Abnormality Classification in Retinal Images , 2018, ArXiv.

[10]  Talha Iqbal,et al.  Generative Adversarial Network for Medical Images (MI-GAN) , 2018, Journal of Medical Systems.

[11]  Sang Jun Park,et al.  Retinal Vessel Segmentation in Fundoscopic Images with Generative Adversarial Networks , 2017, ArXiv.

[12]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[13]  Prabir Kumar Biswas,et al.  Retinal Vessel Segmentation under Extreme Low Annotation: A Generative Adversarial Network Approach , 2018, ArXiv.

[14]  Vincent Lepetit,et al.  Supervised Feature Learning for Curvilinear Structure Segmentation , 2013, MICCAI.

[15]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[16]  Dwarikanath Mahapatra,et al.  Retinal Vasculature Segmentation Using Local Saliency Maps and Generative Adversarial Networks For Image Super Resolution , 2017, ArXiv.

[17]  John T. Guibas,et al.  Synthetic Medical Images from Dual Generative Adversarial Networks , 2017, ArXiv.

[18]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  P. Remagnino,et al.  Retinal image analysis aimed at extraction of vascular structure using linear discriminant classifier , 2013, 2013 International Conference on Computer Medical Applications (ICCMA).

[20]  Max A. Viergever,et al.  Generative Adversarial Networks for Noise Reduction in Low-Dose CT , 2017, IEEE Transactions on Medical Imaging.

[21]  Xuanqin Mou,et al.  Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss , 2017, IEEE Transactions on Medical Imaging.

[22]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Max A. Viergever,et al.  Ridge-based vessel segmentation in color images of the retina , 2004, IEEE Transactions on Medical Imaging.

[24]  Marios S. Pattichis,et al.  Fast Localization and Segmentation of Optic Disk in Retinal Images Using Directional Matched Filtering and Level Sets , 2012, IEEE Transactions on Information Technology in Biomedicine.

[25]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[26]  Tomas Pfister,et al.  Learning from Simulated and Unsupervised Images through Adversarial Training , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[28]  Stephen Lin,et al.  DeepVessel: Retinal Vessel Segmentation via Deep Learning and Conditional Random Field , 2016, MICCAI.

[29]  S. Shankar Sastry,et al.  Characterization and computation of local Nash equilibria in continuous games , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[30]  Roberto Marcondes Cesar Junior,et al.  Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification , 2005, IEEE Transactions on Medical Imaging.

[31]  Nigel M. Bolster,et al.  A review of feature-based retinal image analysis , 2017 .

[32]  Sven Loncaric,et al.  Retinal Vessel Segmentation using Deep Neural Networks , 2015, VISAPP.

[33]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[34]  K Wisaeng,et al.  Automatic detection of exudates in retinal images based on threshold moving average models , 2015, Biofizika.

[35]  Martin Kraus,et al.  Automatic no-reference quality assessment for retinal fundus images using vessel segmentation , 2013, Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems.

[36]  Pabitra Mitra,et al.  Ensemble of Deep Convolutional Neural Networks for Learning to Detect Retinal Vessels in Fundus Images , 2016, ArXiv.

[37]  Won-Ki Jeong,et al.  Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss , 2017, IEEE Transactions on Medical Imaging.

[38]  Hamzah Arof,et al.  Automated Identification of Exudates and Optic Disc Based on Inverse Surface Thresholding , 2012, Journal of Medical Systems.

[39]  Feng Xu,et al.  Smartphone-Based Accurate Analysis of Retinal Vasculature towards Point-of-Care Diagnostics , 2016, Scientific Reports.

[40]  Guo-Jun Qi,et al.  Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities , 2017, International Journal of Computer Vision.

[41]  Joachim M. Buhmann,et al.  Obtaining Consensus Annotations For Retinal Image Segmentation Using Random Forest And Graph Cuts , 2015 .

[42]  A.D. Hoover,et al.  Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response , 2000, IEEE Transactions on Medical Imaging.

[43]  Anil Kumar Tiwari,et al.  Fundus image quality assessment: survey, challenges, and future scope , 2019, IET Image Process..

[44]  Nassir Navab,et al.  GANs for Medical Image Analysis , 2018, Artif. Intell. Medicine.