ipA-MedGAN: Inpainting of Arbitrarily Regions in Medical Modalities

Local deformations in medical modalities are common phenomena due to a multitude of factors such as metallic implants or limited field of views in magnetic resonance imaging (MRI). Completion of the missing or distorted regions is of special interest for automatic image analysis frameworks to enhance post-processing tasks such as segmentation or classification. In this work, we propose a new generative framework for medical image inpainting, titled ipA-MedGAN. It bypasses the limitations of previous frameworks by enabling inpainting of arbitrarily shaped regions without a prior localization of the regions of interest. Thorough qualitative and quantitative comparisons with other inpainting and translational approaches have illustrated the superior performance of the proposed framework for the task of brain MR inpainting.

[1]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[2]  Mohammad Sohel Rahman,et al.  MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation , 2019, Neural Networks.

[3]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[4]  Dacheng Tao,et al.  Perceptual Adversarial Networks for Image-to-Image Transformation , 2017, IEEE Transactions on Image Processing.

[5]  Gerard Lacey,et al.  Automatic Segmentation and Inpainting of Specular Highlights for Endoscopic Imaging , 2010, EURASIP J. Image Video Process..

[6]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[7]  K F King,et al.  Magnetic resonance imaging near metal implants , 2010, Journal of magnetic resonance imaging : JMRI.

[8]  Pavel Vlašánek,et al.  Fuzzy image inpainting aimed to medical images , 2018 .

[9]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Bin Yang,et al.  MedGAN: Medical Image Translation using GANs , 2018, Comput. Medical Imaging Graph..

[11]  Bin Yang,et al.  Adversarial Inpainting of Medical Image Modalities , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[13]  Hiroshi Ishikawa,et al.  Globally and locally consistent image completion , 2017, ACM Trans. Graph..

[14]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[15]  Thomas S. Huang,et al.  Free-Form Image Inpainting With Gated Convolution , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Jianwei Yin,et al.  A Variational Approach to Medical Image Inpainting Based on Mumford-Shah Model , 2007, 2007 International Conference on Service Systems and Service Management.

[17]  Saad Amin,et al.  Medical Image Inpainting with RBF Interpolation Technique , 2016 .

[18]  Sergios Gatidis,et al.  Retrospective Correction Of Rigid And Non-Rigid MR Motion Artifacts Using Gans , 2018, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[19]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[21]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).