FA-GAN: Fused attentive generative adversarial networks for MRI image super-resolution

Highlights • A fused attentive generative adversarial networks framework is proposed for MR image super-resolution.• A combination of channel attention and self-attention is used to calculate the weight parameters of the input features.• Spectral normalization process is introduced to make the discriminator network stabler.• The proposed FA-GAN method is superior to the state-of-the-art reconstruction methods.

[1]  Amir Pasha Mahmoudzadeh,et al.  Interpolation-based super-resolution reconstruction: effects of slice thickness , 2014, Journal of medical imaging.

[2]  Simon K. Warfield,et al.  Robust Super-Resolution Volume Reconstruction From Slice Acquisitions: Application to Fetal Brain MRI , 2010, IEEE Transactions on Medical Imaging.

[3]  Dong Liang,et al.  SRR-Net: A Super-Resolution-Involved Reconstruction Method for High Resolution MR Imaging , 2021, ArXiv.

[4]  Xiaofei Wang,et al.  A Large-Scale Database and a CNN Model for Attention-Based Glaucoma Detection , 2020, IEEE Transactions on Medical Imaging.

[5]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[6]  Hairong Zheng,et al.  DPIR-Net: Direct PET Image Reconstruction Based on the Wasserstein Generative Adversarial Network , 2020, IEEE Transactions on Radiation and Plasma Medical Sciences.

[7]  Dong Liang,et al.  Learning Joint-Sparse Codes for Calibration-Free Parallel MR Imaging , 2018, IEEE Transactions on Medical Imaging.

[8]  Rajib Rana,et al.  Automating Motion Correction in Multishot MRI Using Generative Adversarial Networks , 2018, ArXiv.

[9]  Dong Liang,et al.  DeepcomplexMRI: Exploiting deep residual network for fast parallel MR imaging with complex convolution. , 2020, Magnetic resonance imaging.

[10]  Quoc V. Le,et al.  Attention Augmented Convolutional Networks , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[11]  Xavier Bresson,et al.  An efficient total variation algorithm for super-resolution in fetal brain MRI with adaptive regularization , 2015, NeuroImage.

[12]  Won-Ki Jeong,et al.  Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss , 2017, IEEE Transactions on Medical Imaging.

[13]  Han Zhang,et al.  Self-Attention Generative Adversarial Networks , 2018, ICML.

[14]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer Assisted Intervention , 2017 .

[15]  Forrest N. Iandola,et al.  FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute Clusters , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Jun Xu,et al.  Learning deconvolutional deep neural network for high resolution medical image reconstruction , 2018, Inf. Sci..

[17]  Sergios Gatidis,et al.  Retrospective Correction Of Rigid And Non-Rigid MR Motion Artifacts Using Gans , 2018, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[18]  Quoc V. Le,et al.  Searching for Activation Functions , 2018, arXiv.

[19]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Jean-Yves Tourneret,et al.  Fast Single Image Super-Resolution , 2015, ArXiv.

[21]  Norberto Malpica,et al.  Single-image super-resolution of brain MR images using overcomplete dictionaries , 2013, Medical Image Anal..

[22]  Thomas J. Fuchs,et al.  DeepPET: A deep encoder–decoder network for directly solving the PET image reconstruction inverse problem , 2018, Medical Image Anal..

[23]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Kourosh Jafari-Khouzani,et al.  MRI Upsampling Using Feature-Based Nonlocal Means Approach , 2014, IEEE Transactions on Medical Imaging.

[25]  Jun Fu,et al.  Dual Attention Network for Scene Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Shihui Ying,et al.  Super-resolution reconstruction of MR image with a novel residual learning network algorithm , 2018, Physics in medicine and biology.

[27]  Stephen Lin,et al.  An Empirical Study of Spatial Attention Mechanisms in Deep Networks , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[28]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Zhiyuan Liu,et al.  Deep Learning Based Framework for Direct Reconstruction of PET Images , 2019, MICCAI.

[30]  Luca Ambrogioni,et al.  Generative adversarial networks for reconstructing natural images from brain activity , 2017, NeuroImage.

[31]  Ting Zhao,et al.  Pyramid Feature Attention Network for Saliency Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[33]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[34]  Huafeng Liu,et al.  Direct delineation of myocardial infarction without contrast agents using a joint motion feature learning architecture , 2018, Medical Image Anal..

[35]  Dong Liang,et al.  DIMENSION: Dynamic MR imaging with both k‐space and spatial prior knowledge obtained via multi‐supervised network training , 2018, NMR in biomedicine.

[36]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Dong Liang,et al.  A Comparative Study of CNN-Based Super-Resolution Methods in MRI Reconstruction , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[38]  Haoqian Wang,et al.  Single MR image super-resolution via mixed self-similarity attention network , 2021, Medical Imaging.

[39]  Wanqing Li,et al.  Fast single image super-resolution using estimated low-frequency k-space data in MRI. , 2017, Magnetic resonance imaging.

[40]  Amod Jog,et al.  Improving magnetic resonance resolution with supervised learning , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[41]  Shihui Ying,et al.  MR Image Super-Resolution via Wide Residual Networks With Fixed Skip Connection , 2019, IEEE Journal of Biomedical and Health Informatics.