Generative adversarial networks and adversarial methods in biomedical image analysis

Generative adversarial networks (GANs) and other adversarial methods are based on a game-theoretical perspective on joint optimization of two neural networks as players in a game. Adversarial techniques have been extensively used to synthesize and analyze biomedical images. We provide an introduction to GANs and adversarial methods, with an overview of biomedical image analysis tasks that have benefited from such methods. We conclude with a discussion of strengths and limitations of adversarial methods in biomedical image analysis, and propose potential future research directions.

[1]  Ulas Bagci,et al.  How to fool radiologists with generative adversarial networks? A visual turing test for lung cancer diagnosis , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[2]  Frank Niemeyer,et al.  Exploring the Potential of Generative Adversarial Networks for Synthesizing Radiological Images of the Spine to be Used in In Silico Trials , 2018, Front. Bioeng. Biotechnol..

[3]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[4]  Lin Yang,et al.  Deep Adversarial Networks for Biomedical Image Segmentation Utilizing Unannotated Images , 2017, MICCAI.

[5]  Subarna Tripathi,et al.  Precise Recovery of Latent Vectors from Generative Adversarial Networks , 2017, ICLR.

[6]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[7]  Tim Leiner,et al.  Blood Vessel Geometry Synthesis using Generative Adversarial Networks , 2018, ArXiv.

[8]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[9]  Konstantinos Kamnitsas,et al.  Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation , 2017, IEEE Transactions on Medical Imaging.

[10]  Xiao Han,et al.  MR‐based synthetic CT generation using a deep convolutional neural network method , 2017, Medical physics.

[11]  Pascal Fua,et al.  Scalable Unsupervised Domain Adaptation for Electron Microscopy , 2016, MICCAI.

[12]  Tao Xu,et al.  SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation , 2017, Neuroinformatics.

[13]  Dumitru Erhan,et al.  Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Won-Ki Jeong,et al.  Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss , 2017, IEEE Transactions on Medical Imaging.

[15]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[16]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Konstantinos Kamnitsas,et al.  Unsupervised domain adaptation in brain lesion segmentation with adversarial networks , 2016, IPMI.

[18]  Mitko Veta,et al.  Adversarial Training and Dilated Convolutions for Brain MRI Segmentation , 2017, DLMIA/ML-CDS@MICCAI.

[19]  Max A. Viergever,et al.  Generative Adversarial Networks for Noise Reduction in Low-Dose CT , 2017, IEEE Transactions on Medical Imaging.

[20]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[21]  Hyunsoo Kim,et al.  Learning to Discover Cross-Domain Relations with Generative Adversarial Networks , 2017, ICML.

[22]  Shunxing Bao,et al.  Adversarial synthesis learning enables segmentation without target modality ground truth , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[23]  Jaime S. Cardoso,et al.  Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support , 2017, Lecture Notes in Computer Science.

[24]  Tomas Pfister,et al.  Learning from Simulated and Unsupervised Images through Adversarial Training , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[26]  Mark Sandler,et al.  CycleGAN, a Master of Steganography , 2017, ArXiv.

[27]  Jong Chul Ye,et al.  A deep convolutional neural network using directional wavelets for low‐dose X‐ray CT reconstruction , 2016, Medical physics.

[28]  Dinggang Shen,et al.  3D conditional generative adversarial networks for high-quality PET image estimation at low dose , 2018, NeuroImage.

[29]  Nassir Navab,et al.  Domain and Geometry Agnostic CNNs for Left Atrium Segmentation in 3D Ultrasound , 2018, MICCAI.

[30]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[31]  Xuanqin Mou,et al.  Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss , 2017, IEEE Transactions on Medical Imaging.

[32]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[33]  Ki Hwan Kim,et al.  Improving resolution of MR images with an adversarial network incorporating images with different contrast , 2018, Medical physics.

[34]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[35]  Sina Honari,et al.  Distribution Matching Losses Can Hallucinate Features in Medical Image Translation , 2018, MICCAI.

[36]  Jelmer M. Wolterink,et al.  MR-Only Brain Radiation Therapy: Dosimetric Evaluation of Synthetic CTs Generated by a Dilated Convolutional Neural Network. , 2018, International journal of radiation oncology, biology, physics.

[37]  Ping Tan,et al.  DualGAN: Unsupervised Dual Learning for Image-to-Image Translation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[38]  Georg Langs,et al.  Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery , 2017, IPMI.

[39]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[40]  David Pfau,et al.  Unrolled Generative Adversarial Networks , 2016, ICLR.

[41]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[42]  Camille Couprie,et al.  Semantic Segmentation using Adversarial Networks , 2016, NIPS 2016.

[43]  Klaus H. Maier-Hein,et al.  Adversarial Networks for the Detection of Aggressive Prostate Cancer , 2017, ArXiv.

[44]  Trevor Darrell,et al.  Adversarial Feature Learning , 2016, ICLR.

[45]  Dong Soo Lee,et al.  Generation of Structural MR Images from Amyloid PET: Application to MR-Less Quantification , 2017, The Journal of Nuclear Medicine.

[46]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[47]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Ender Konukoglu,et al.  Visual Feature Attribution Using Wasserstein GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[49]  Yochai Blau,et al.  The Perception-Distortion Tradeoff , 2017, CVPR.

[50]  James M. Brown,et al.  High-resolution medical image synthesis using progressively grown generative adversarial networks , 2018, ArXiv.

[51]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[52]  Hao Chen,et al.  Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss , 2018, IJCAI.

[53]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[54]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.

[55]  Dimitris N. Metaxas,et al.  StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[56]  Sotirios A. Tsaftaris,et al.  Adversarial Image Synthesis for Unpaired Multi-modal Cardiac Data , 2017, SASHIMI@MICCAI.

[57]  Tanveer F. Syeda-Mahmood,et al.  Semi-supervised learning with generative adversarial networks for chest X-ray classification with ability of data domain adaptation , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[58]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[59]  Hao Chen,et al.  Semantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-ray Segmentation , 2018, MLMI@MICCAI.

[60]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[61]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[62]  Song Han,et al.  Deep Generative Adversarial Networks for Compressed Sensing Automates MRI , 2017, ArXiv.

[63]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[64]  Faisal Mahmood,et al.  Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training , 2017, IEEE Transactions on Medical Imaging.

[65]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..