Multimodal Image-to-Image Translation for Generation of Gastritis Images

We present a new multimodal image-to-image translation model for the generation of gastritis images using X-ray and blood inspection results. In clinical situations, clinicians estimate the prognosis of the target disease by considering multiple inspection results. Similarly, we take a multimodal approach in the task of gastric cancer risk prediction. Visual characteristics of the gastric X-ray image and blood index values are highly related in the evaluation of gastric cancer risk. If we can generate a prediction image from blood index values, it contributes to the clinicians’ sophisticated and integrated diagnosis. Hence, we learn a model that can map non-gastritis images to gastritis images based on the blood index values. Although this is a challenging multimodal task in medical image analysis, experimental results showed the effectiveness of our model.

[1]  N. Yoshimura,et al.  Progression of chronic atrophic gastritis associated with Helicobacter pylori infection increases risk of gastric cancer , 2004, International journal of cancer.

[2]  M. Mori,et al.  Analysis of ABC (D) stratification for screening patients with gastric cancer. , 2011, World journal of gastroenterology.

[3]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[4]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[5]  Miki Haseyama,et al.  Gastritis Detection from Gastric X-Ray Images Via Fine-Tuning of Patch-Based Deep Convolutional Neural Network , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[6]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[7]  Yong-Jin Liu,et al.  CartoonGAN: Generative Adversarial Networks for Photo Cartoonization , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Max A. Viergever,et al.  Generative Adversarial Networks for Noise Reduction in Low-Dose CT , 2017, IEEE Transactions on Medical Imaging.

[9]  Georg Langs,et al.  Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery , 2017, IPMI.

[10]  Trevor Darrell,et al.  Adversarial Feature Learning , 2016, ICLR.

[11]  Shahrokh Valaee,et al.  Generalization of Deep Neural Networks for Chest Pathology Classification in X-Rays Using Generative Adversarial Networks , 2017, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[13]  Yoshua Bengio,et al.  Improving Generative Adversarial Networks with Denoising Feature Matching , 2016, ICLR.

[14]  K. Kichikawa,et al.  Correlation between the ABC classification and radiological findings for assessing gastric cancer risk , 2015, Japanese Journal of Radiology.

[15]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[16]  Miki Haseyama,et al.  Synthetic Gastritis Image Generation via Loss Function-Based Conditional PGGAN , 2019, IEEE Access.

[17]  Miki Haseyama,et al.  Anonymous Gastritis Image Generation via Adversarial Learning from Gastric X-Ray Images , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[18]  Zhe Gan,et al.  AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  George Lee,et al.  Image analysis and machine learning in digital pathology: Challenges and opportunities , 2016, Medical Image Anal..

[20]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[22]  Yosuke Tsuji,et al.  Comparative analysis of upper gastrointestinal endoscopy, double-contrast upper gastrointestinal barium X-ray radiography, and the titer of serum anti-Helicobacter pylori IgG focusing on the diagnosis of atrophic gastritis , 2016, Gastric Cancer.

[23]  Ya Tu,et al.  Digital Signal Modulation Classification With Data Augmentation Using Generative Adversarial Nets in Cognitive Radio Networks , 2018, IEEE Access.