Exploiting the Generative Adversarial Network Approach to Create a Synthetic Topography Corneal Image

Corneal diseases are the most common eye disorders. Deep learning techniques are used to perform automated diagnoses of cornea. Deep learning networks require large-scale annotated datasets, which is conceded as a weakness of deep learning. In this work, a method for synthesizing medical images using conditional generative adversarial networks (CGANs), is presented. It also illustrates how produced medical images may be utilized to enrich medical data, improve clinical decisions, and boost the performance of the conventional neural network (CNN) for medical image diagnosis. The study includes using corneal topography captured using a Pentacam device from patients with corneal diseases. The dataset contained 3448 different corneal images. Furthermore, it shows how an unbalanced dataset affects the performance of classifiers, where the data are balanced using the resampling approach. Finally, the results obtained from CNN networks trained on the balanced dataset are compared to those obtained from CNN networks trained on the imbalanced dataset. For performance, the system estimated the diagnosis accuracy, precision, and F1-score metrics. Lastly, some generated images were shown to an expert for evaluation and to see how well experts could identify the type of image and its condition. The expert recognized the image as useful for medical diagnosis and for determining the severity class according to the shape and values, by generating images based on real cases that could be used as new different stages of illness between healthy and unhealthy patients.

[1]  Samer Kais Jameel,et al.  Face Identification System Based on Synthesizing Realistic Image using Edge-Aided GANs , 2021, Comput. J..

[2]  Jafar Majidpour,et al.  Generating Spectrum Images from Different Types - Visible, Thermal, and Infrared Based on Autoencoder Architecture (GVTI-AE) , 2021, International Journal of Image and Graphics.

[3]  Samer Kais Jameel,et al.  Local information pattern descriptor for corneal diseases diagnosis , 2021 .

[4]  M. Pyingkodi,et al.  Artificial intelligence applications in different imaging modalities for corneal topography. , 2021, Survey of ophthalmology.

[5]  Zhuowen Tu,et al.  Co-Scale Conv-Attentional Image Transformers , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Nebras H. Ghaeb,et al.  SWFT: Subbands wavelet for local features transform descriptor for corneal diseases diagnosis , 2021, Turkish J. Electr. Eng. Comput. Sci..

[7]  Quanfu Fan,et al.  CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[8]  Mesut Toğaçar,et al.  Intelligent skin cancer detection applying autoencoder, MobileNetV2 and spiking neural networks , 2021 .

[9]  A. Ljubimov,et al.  Systemic diseases and the cornea. , 2021, Experimental eye research.

[10]  Yuxi Wang,et al.  Multi-Domain Image-to-Image Translation via a Unified Circular Framework , 2020, IEEE Trans. Image Process..

[11]  Samer Kais Jameel,et al.  Machine Learning Techniques for Corneal Diseases Diagnosis: A Survey , 2020, Int. J. Image Graph..

[12]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[13]  Stephen Lin,et al.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[14]  Girish Kumar,et al.  Synthetic data augmentation for surface defect detection and classification using deep learning , 2020, Journal of Intelligent Manufacturing.

[15]  V. Wong,et al.  The Asian Pacific Association for the Study of the Liver clinical practice guidelines for the diagnosis and management of metabolic associated fatty liver disease , 2020, Hepatology International.

[16]  Tsung-Yi Ho,et al.  Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resources , 2020, ICML.

[17]  E. Chang,et al.  MRI Cross-Modality Image-to-Image Translation , 2020, Scientific Reports.

[18]  Eli Konen,et al.  Creating Artificial Images for Radiology Applications Using Generative Adversarial Networks (GANs) - A Systematic Review. , 2020, Academic radiology.

[19]  Yi Yang,et al.  Random Erasing Data Augmentation , 2017, AAAI.

[20]  Paul Babyn,et al.  Generative Adversarial Network in Medical Imaging: A Review , 2018, Medical Image Anal..

[21]  Wouter M. Kouw,et al.  CT image segmentation of bone for medical additive manufacturing using a convolutional neural network , 2018, Comput. Biol. Medicine.

[22]  Jos J. Rozema,et al.  Corneal modeling and Keratoconus identification , 2018 .

[23]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Tao Xu,et al.  SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation , 2017, Neuroinformatics.

[26]  Tanveer F. Syeda-Mahmood,et al.  Towards an Efficient Way of Building Annotated Medical Image Collections for Big Data Studies , 2017, CVII-STENT/LABELS@MICCAI.

[27]  Tareq Abed Mohammed,et al.  Understanding of a convolutional neural network , 2017, 2017 International Conference on Engineering and Technology (ICET).

[28]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[29]  Georg Langs,et al.  Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery , 2017, IPMI.

[30]  Pedro Costa,et al.  Towards Adversarial Retinal Image Synthesis , 2017, ArXiv.

[31]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[32]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Jennifer Alvén,et al.  Improving Multi-Atlas Segmentation Methods for Medical Images , 2017 .

[35]  Yuan Li,et al.  SCAN: Structure Correcting Adversarial Network for Chest X-rays Organ Segmentation , 2017, ArXiv.

[36]  Yann LeCun,et al.  Energy-based Generative Adversarial Network , 2016, ICLR.

[37]  Bernardo T Lopes,et al.  Detection of ectatic corneal diseases based on pentacam. , 2016, Zeitschrift fur medizinische Physik.

[38]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Carlos Guestrin,et al.  Vector Machine , 2016 .

[40]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[41]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[42]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[43]  Xiaogang Wang,et al.  Medical image classification with convolutional neural network , 2014, 2014 13th International Conference on Control Automation Robotics & Vision (ICARCV).

[44]  Francesco Versaci,et al.  Use of a support vector machine for keratoconus and subclinical keratoconus detection by topographic and tomographic data. , 2012, Ophthalmology.

[45]  Djemel Ziou,et al.  Image Quality Metrics: PSNR vs. SSIM , 2010, 2010 20th International Conference on Pattern Recognition.

[46]  H. Hashemi,et al.  Day to Day Clinically Relevant Corneal Elevation, Thickness, and Curvature Parameters Using the Orbscan II Scanning Slit Topographer and the Pentacam Scheimpflug Imaging Device , 2010, Middle East African journal of ophthalmology.

[47]  Xue Dong Yang,et al.  Existing and emerging image quality metrics , 2005, Canadian Conference on Electrical and Computer Engineering, 2005..

[48]  Pavel Slavík,et al.  Evaluation of two principal approaches to objective image quality assessment , 2004, Proceedings. Eighth International Conference on Information Visualisation, 2004. IV 2004..

[49]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[50]  Djemel Ziou,et al.  Contextual and non-contextual performance evaluation of edge detectors , 2000, Pattern Recognit. Lett..

[51]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[52]  M. Kamel,et al.  An information theoretic image-quality measure , 1998, Conference Proceedings. IEEE Canadian Conference on Electrical and Computer Engineering (Cat. No.98TH8341).